Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 21.414564101s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:32:16 UTC) orbiter warning: context deadline exceeded Duration: 21.414590711s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X....X...........X........X.X...XX.X..X.............X.......] duration: 13.085932925s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:32:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 13.085959536s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX] duration: 5.000977911s trace: https://jaeger.dev.storj.io/trace/629cdfa83a28f146 FAIL (2026-01-09 06:31:59 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000994402s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................X.....X..X.] duration: 7.517681729s trace: https://jaeger.dev.storj.io/trace/42a8af45cb4262e FAIL (2026-01-09 06:31:25 UTC) orbiter warning: context deadline exceeded Duration: 7.517706447s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 28.705161238s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:31:19 UTC) orbiter warning: context deadline exceeded Duration: 28.705194331s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................X.....X..X] duration: 30.001724458s trace: https://jaeger.dev.storj.io/trace/42a8af45cb4262e FAIL (2026-01-09 06:31:04 UTC) orbiter warning: context deadline exceeded Duration: 30.001747723s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XX....X...........X........X.X...XX.X..X.............X......] duration: 16.072652124s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:30:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 16.072667773s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X] duration: 5.001903125s trace: https://jaeger.dev.storj.io/trace/55a9b24d1f7fc151 FAIL (2026-01-09 06:30:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001928214s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.....X..] duration: 7.083336488s trace: https://jaeger.dev.storj.io/trace/6263fa5eb2bad249 FAIL (2026-01-09 06:30:16 UTC) orbiter warning: context deadline exceeded Duration: 7.083369342s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 17.972880721s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:30:14 UTC) orbiter warning: context deadline exceeded Duration: 17.972902692s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.....X.] duration: 16.984013387s trace: https://jaeger.dev.storj.io/trace/6263fa5eb2bad249 FAIL (2026-01-09 06:29:57 UTC) orbiter warning: context deadline exceeded Duration: 16.984134897s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.] duration: 3.587176716s trace: https://jaeger.dev.storj.io/trace/52f716256ac9475a FAIL (2026-01-09 06:29:40 UTC) orbiter warning: context deadline exceeded Duration: 3.587196293s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.XX....X...........X........X.X...XX.X..X.............X.....] duration: 6.171727083s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:29:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 6.17174675s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 13.908680392s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:29:10 UTC) orbiter warning: context deadline exceeded Duration: 13.908699018s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.....X] duration: 30.000708637s trace: https://jaeger.dev.storj.io/trace/6263fa5eb2bad249 FAIL (2026-01-09 06:29:05 UTC) orbiter warning: context deadline exceeded Duration: 30.000736169s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..XX....X...........X........X.X...XX.X..X.............X....] duration: 5.852084875s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:28:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 5.852112417s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 16.61082998s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:28:53 UTC) orbiter warning: context deadline exceeded Duration: 16.610841392s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 19.584699923s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:28:37 UTC) orbiter warning: context deadline exceeded Duration: 19.584725353s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001680137s trace: https://jaeger.dev.storj.io/trace/52f716256ac9475a FAIL (2026-01-09 06:28:31 UTC) orbiter warning: context deadline exceeded Duration: 5.001697281s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...XX....X...........X........X.X...XX.X..X.............X...] duration: 5.855043576s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:28:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 5.855075508s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 17.166448355s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:27:57 UTC) orbiter warning: context deadline exceeded Duration: 17.166468813s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 7.101247937s trace: https://jaeger.dev.storj.io/trace/6467006c814382be FAIL (2026-01-09 06:27:42 UTC) orbiter warning: context deadline exceeded Duration: 7.101273195s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001381789s trace: https://jaeger.dev.storj.io/trace/55d892c03a644d3b FAIL (2026-01-09 06:27:29 UTC) orbiter warning: context deadline exceeded Duration: 5.001404874s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....XX....X...........X........X.X...XX.X..X.............X..] duration: 6.22546309s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:27:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 6.225487315s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 15.795291779s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:27:01 UTC) orbiter warning: context deadline exceeded Duration: 15.79532844s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 22.130429164s trace: https://jaeger.dev.storj.io/trace/6467006c814382be FAIL (2026-01-09 06:26:54 UTC) orbiter warning: context deadline exceeded Duration: 22.130447868s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001688447s trace: https://jaeger.dev.storj.io/trace/43ab025d4c3ae77f FAIL (2026-01-09 06:26:29 UTC) orbiter warning: context deadline exceeded Duration: 5.001712204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....XX....X...........X........X.X...XX.X..X.............X.] duration: 6.026854627s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:26:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 6.026898531s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 20.473266311s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:26:04 UTC) orbiter warning: context deadline exceeded Duration: 20.473294837s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......XX....X...........X........X.X...XX.X..X.............X] duration: 30.00233965s trace: https://jaeger.dev.storj.io/trace/1b142191e714a682 FAIL (2026-01-09 06:25:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Jan 9 06:25:28.981382447 (830h56m28.852465492s) [upload=726813]: Got next segment. Inline: false Jan 9 06:25:28.981415970 (830h56m28.852499005s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:25:28.981423775 (830h56m28.85250681s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:25:29.224667635 (830h56m29.09575067s) [seg_pos={0 0}, upload=726813]: Begin upload segment... Jan 9 06:25:29.224709434 (830h56m29.095792469s) [seg_pos={0 0}, upload=726813]: Done begin upload segment. Jan 9 06:25:29.224738079 (830h56m29.095821114s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.224760772 (830h56m29.095843808s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.225607214 (830h56m29.096690259s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226128276 (830h56m29.097211321s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.226563504 (830h56m29.097646549s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227029382 (830h56m29.098112427s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227488697 (830h56m29.098571742s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.227913094 (830h56m29.09899614s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.229300486 (830h56m29.100383532s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.230589210 (830h56m29.101672245s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.231759238 (830h56m29.102842283s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.233216673 (830h56m29.104299708s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.234285218 (830h56m29.105368253s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.235443083 (830h56m29.106526118s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.236604554 (830h56m29.107687599s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:29.237935208 (830h56m29.109018253s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Uploading piece... Jan 9 06:25:30.840810166 (830h56m30.711893201s) [upload=726813]: Next returned nil segment Jan 9 06:25:30.840844481 (830h56m30.711927516s) [upload=726813]: Waiting for error group managing segments... Jan 9 06:25:31.225348859 (830h56m31.096431904s) [noise=true , address=145.79.227.49:4339 , piece_id=3UHWCV4ZEU7PS2267AUNC6YW3DJFHJVGRMXLKWIQUDFEFFI6KFDQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.229698642 (830h56m31.100781687s) [noise=true , address=145.79.227.44:4194 , piece_id=EJQGFZ6SPLTOCISFRDYVOM2ZOMK7ESMIN3PZZBKKIVSPVNJCHQVQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.230277064 (830h56m31.101360099s) [noise=true , address=145.79.227.16:4909 , piece_id=NAMHPAB5XT5DG56XAOW5JNZVKGAOZ47PKILITCVKZAMM3DTQFLUQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.235910009 (830h56m31.106993064s) [noise=true , address=145.79.227.15:4876 , piece_id=WM6DP7S7WCT2FBAZ7CXGBKMLFUKH6GXIYJZ62BVD3YD4DEJPOENA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237083183 (830h56m31.108166218s) [noise=true , address=145.79.227.35:4481 , piece_id=UTVWVX2RN44IO7BFI7OMX6NDGYBUIJXFQCVCDE2IUKJXLBRRHDBQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.237828502 (830h56m31.108911547s) [noise=true , address=145.79.227.17:4970 , piece_id=L24A7AFCU2ZSQYOFII5R6MPVHCJB2MBHWGCIDZJ2S3TTXQQU6JOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.248246419 (830h56m31.119329464s) [noise=true , address=145.79.227.23:4545 , piece_id=GFU7CMPNEIUVX2Z6WISWBYWMIUYLG2PACT2WMKTQPSZ6IC3AVKQA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.253091415 (830h56m31.12417446s) [noise=true , address=145.79.227.9:4367 , piece_id=VSHWUVN4XJRGGO7CSAWZZFIALT4IJRDZYMGAAJN7VAZFKI2AIVFQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.258888663 (830h56m31.129971698s) [noise=true , address=145.79.227.20:4439 , piece_id=3SXWPJ3MWNWRM46FMWEEC5B2RYONOBVNVTKXV6G6WUFHAW3ZWD3A, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.266242566 (830h56m31.137325601s) [noise=true , address=145.79.227.29:4943 , piece_id=OATVISXVMPP3PYYMACHVVNVNKF2VUEI2RYICTDKRVM5AEGV4VUHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.537392222 (830h56m31.408475267s) [noise=true , address=77.90.31.19:4844 , piece_id=EOR56DT7ASUT2333OOEHXF72BC23T72PXBEWF6H33O6FZ6YPEBZA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.542760924 (830h56m31.41384397s) [noise=true , address=77.90.31.11:4556 , piece_id=R5JCUSY4FMT4SU6Y6GGVF2OMCKI24ZNNC7DXL34HICLTBD2PLQXQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604519872 (830h56m31.475602907s) [noise=true , address=77.90.31.10:4519 , piece_id=ODJ4V2ODHW5JHSUA22ZUSJTAAX6D3GCEJZW625YNRVQ3EGNJAAOA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: <nil> Jan 9 06:25:31.604566651 (830h56m31.475649686s) [seg_pos={0 0}, upload=726813]: Segment reached optimal threshold of 13 pieces. Jan 9 06:25:31.604804784 (830h56m31.475887829s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t): protocol: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44300->45.91.116.7:4012: use of closed network connection Jan 9 06:25:31.604812489 (830h56m31.475895534s) [noise=true , address=45.91.116.7:4012 , piece_id=DBYTBZWQWQZU25A7UV7JVKD62FE7Q4AE5A7VDESZPLMHS4VLNVHA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.604923251 (830h56m31.476006296s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43796->45.91.116.6:4976: use of closed network connection Jan 9 06:25:31.604928591 (830h56m31.476011626s) [noise=true , address=45.91.116.6:4976 , piece_id=WZCYHGFRY2JOYQ2QX4SZZTNRSMAOLTRYX2X3CSQI67MC6AMZJXRA, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605115686 (830h56m31.476198721s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Jan 9 06:25:31.605140113 (830h56m31.476223148s) [noise=true , address=45.91.116.9:4264 , piece_id=RZMXB2KNR6KZJD4KTS7NKPPTU4YDK3T6JEEDCVL6Z34AVYHLD6NQ, seg_pos={0 0}, upload=726813]: Long tail cancellation due to optimal threshold Jan 9 06:25:31.605186992 (830h56m31.476270037s) [seg_pos={0 0}, upload=726813]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:25:31.605306310 (830h56m31.476389345s) [upload=726813]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:25:31.605312381 (830h56m31.476395426s) [upload=726813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:25:31.887743968 (830h56m31.758827004s) [upload=726813]: Done uploading segments... Jan 9 06:25:31.887763546 (830h56m31.758846591s) [upload=726813]: Upload finished. err: <nil> Duration: 30.002417257s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 24.825374579s trace: https://jaeger.dev.storj.io/trace/6467006c814382be FAIL (2026-01-09 06:25:53 UTC) orbiter warning: context deadline exceeded Duration: 24.825400268s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001070445s trace: https://jaeger.dev.storj.io/trace/bfac4de407f789a FAIL (2026-01-09 06:25:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001095694s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 16.426682466s trace: https://jaeger.dev.storj.io/trace/6467006c814382be FAIL (2026-01-09 06:25:11 UTC) orbiter warning: context deadline exceeded Duration: 16.426708887s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 16.888123149s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:25:10 UTC) orbiter warning: context deadline exceeded Duration: 16.888138317s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......XX....X...........X........X.X...XX.X..X.............] duration: 6.178220461s trace: https://jaeger.dev.storj.io/trace/767dc81c2be0328 FAIL (2026-01-09 06:25:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ) context deadline exceeded; piecestore: (Node ID: 1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, Piece ID: OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A) context deadline exceeded ======LOGS====== 9 06:14:44.133209750 (830h45m44.004292795s) [upload=726663]: Starting upload Jan 9 06:14:44.133266649 (830h45m44.004349684s) [upload=726663]: Done starting upload Jan 9 06:14:44.133397809 (830h45m44.004480844s) [upload=726663]: Uploading segments... Jan 9 06:14:44.133499823 (830h45m44.004582868s) [upload=726663]: Got next segment. Inline: false Jan 9 06:14:44.133570127 (830h45m44.004653162s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:14:44.133577421 (830h45m44.004660456s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:14:44.376950314 (830h45m44.248033349s) [seg_pos={0 0}, upload=726663]: Begin upload segment... Jan 9 06:14:44.377003566 (830h45m44.248086601s) [seg_pos={0 0}, upload=726663]: Done begin upload segment. Jan 9 06:14:44.377040186 (830h45m44.248123231s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377085161 (830h45m44.248168206s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377548564 (830h45m44.248631609s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377722315 (830h45m44.24880536s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378124310 (830h45m44.249207355s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378312648 (830h45m44.249395683s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378696029 (830h45m44.249779064s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379067125 (830h45m44.25015016s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379131979 (830h45m44.250215024s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379615760 (830h45m44.250698805s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379960707 (830h45m44.251043742s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380240650 (830h45m44.251323695s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380602809 (830h45m44.251685844s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380827668 (830h45m44.251910703s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381186972 (830h45m44.252270007s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381350423 (830h45m44.252433459s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:46.084711067 (830h45m45.955794102s) [upload=726663]: Next returned nil segment Jan 9 06:14:46.084724584 (830h45m45.955807619s) [upload=726663]: Waiting for error group managing segments... Jan 9 06:14:46.371445789 (830h45m46.242528824s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.372939333 (830h45m46.244022378s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385195200 (830h45m46.256278235s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385700692 (830h45m46.256783737s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.390346538 (830h45m46.261429573s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.398302218 (830h45m46.269385263s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.418777034 (830h45m46.289860069s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.445630346 (830h45m46.316713381s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.554868320 (830h45m46.425951355s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.735038076 (830h45m46.606121121s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.752429242 (830h45m46.623512287s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.777515640 (830h45m46.648598685s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780170044 (830h45m46.651253089s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780182227 (830h45m46.651265272s) [seg_pos={0 0}, upload=726663]: Segment reached optimal threshold of 13 pieces. Jan 9 06:14:46.780328837 (830h45m46.651411872s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780336381 (830h45m46.651419416s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780487759 (830h45m46.651570804s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780498510 (830h45m46.651581545s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780537825 (830h45m46.65162086s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection Jan 9 06:14:46.780544738 (830h45m46.651627773s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780559826 (830h45m46.651642861s) [seg_pos={0 0}, upload=726663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:14:46.780622375 (830h45m46.65170541s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:14:46.780626643 (830h45m46.651709688s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:14:47.082429286 (830h45m46.953512321s) [upload=726663]: Done uploading segments... Jan 9 06:14:47.082454053 (830h45m46.953537098s) [upload=726663]: Upload finished. err: <nil> Duration: 6.178250938s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 22.246550503s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:24:48 UTC) orbiter warning: context deadline exceeded Duration: 22.246575711s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 26.876455282s trace: https://jaeger.dev.storj.io/trace/6467006c814382be FAIL (2026-01-09 06:24:43 UTC) orbiter warning: context deadline exceeded Duration: 26.876478206s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000399999s trace: https://jaeger.dev.storj.io/trace/5cb2e686807af1fb FAIL (2026-01-09 06:24:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000423614s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 15.690853324s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:23:56 UTC) orbiter warning: context deadline exceeded Duration: 15.690876318s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........XX....X...........X........X.X...XX.X..X............] duration: 5.829554865s trace: https://jaeger.dev.storj.io/trace/767dc81c2be0328 FAIL (2026-01-09 06:23:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ) context deadline exceeded; piecestore: (Node ID: 1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, Piece ID: OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A) context deadline exceeded ======LOGS====== 9 06:14:44.133209750 (830h45m44.004292795s) [upload=726663]: Starting upload Jan 9 06:14:44.133266649 (830h45m44.004349684s) [upload=726663]: Done starting upload Jan 9 06:14:44.133397809 (830h45m44.004480844s) [upload=726663]: Uploading segments... Jan 9 06:14:44.133499823 (830h45m44.004582868s) [upload=726663]: Got next segment. Inline: false Jan 9 06:14:44.133570127 (830h45m44.004653162s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:14:44.133577421 (830h45m44.004660456s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:14:44.376950314 (830h45m44.248033349s) [seg_pos={0 0}, upload=726663]: Begin upload segment... Jan 9 06:14:44.377003566 (830h45m44.248086601s) [seg_pos={0 0}, upload=726663]: Done begin upload segment. Jan 9 06:14:44.377040186 (830h45m44.248123231s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377085161 (830h45m44.248168206s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377548564 (830h45m44.248631609s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377722315 (830h45m44.24880536s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378124310 (830h45m44.249207355s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378312648 (830h45m44.249395683s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378696029 (830h45m44.249779064s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379067125 (830h45m44.25015016s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379131979 (830h45m44.250215024s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379615760 (830h45m44.250698805s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379960707 (830h45m44.251043742s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380240650 (830h45m44.251323695s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380602809 (830h45m44.251685844s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380827668 (830h45m44.251910703s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381186972 (830h45m44.252270007s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381350423 (830h45m44.252433459s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:46.084711067 (830h45m45.955794102s) [upload=726663]: Next returned nil segment Jan 9 06:14:46.084724584 (830h45m45.955807619s) [upload=726663]: Waiting for error group managing segments... Jan 9 06:14:46.371445789 (830h45m46.242528824s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.372939333 (830h45m46.244022378s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385195200 (830h45m46.256278235s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385700692 (830h45m46.256783737s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.390346538 (830h45m46.261429573s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.398302218 (830h45m46.269385263s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.418777034 (830h45m46.289860069s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.445630346 (830h45m46.316713381s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.554868320 (830h45m46.425951355s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.735038076 (830h45m46.606121121s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.752429242 (830h45m46.623512287s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.777515640 (830h45m46.648598685s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780170044 (830h45m46.651253089s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780182227 (830h45m46.651265272s) [seg_pos={0 0}, upload=726663]: Segment reached optimal threshold of 13 pieces. Jan 9 06:14:46.780328837 (830h45m46.651411872s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780336381 (830h45m46.651419416s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780487759 (830h45m46.651570804s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780498510 (830h45m46.651581545s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780537825 (830h45m46.65162086s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection Jan 9 06:14:46.780544738 (830h45m46.651627773s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780559826 (830h45m46.651642861s) [seg_pos={0 0}, upload=726663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:14:46.780622375 (830h45m46.65170541s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:14:46.780626643 (830h45m46.651709688s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:14:47.082429286 (830h45m46.953512321s) [upload=726663]: Done uploading segments... Jan 9 06:14:47.082454053 (830h45m46.953537098s) [upload=726663]: Upload finished. err: <nil> Duration: 5.82959912s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000975408s trace: https://jaeger.dev.storj.io/trace/6467006c814382be FAIL (2026-01-09 06:23:40 UTC) orbiter warning: context deadline exceeded Duration: 30.000999444s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 18.221917568s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:23:34 UTC) orbiter warning: context deadline exceeded Duration: 18.221943288s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........XX....X...........X........X.X...XX.X..X...........] duration: 29.697559382s trace: https://jaeger.dev.storj.io/trace/767dc81c2be0328 FAIL (2026-01-09 06:23:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ) context deadline exceeded; piecestore: (Node ID: 1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, Piece ID: OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A) context deadline exceeded ======LOGS====== 9 06:14:44.133209750 (830h45m44.004292795s) [upload=726663]: Starting upload Jan 9 06:14:44.133266649 (830h45m44.004349684s) [upload=726663]: Done starting upload Jan 9 06:14:44.133397809 (830h45m44.004480844s) [upload=726663]: Uploading segments... Jan 9 06:14:44.133499823 (830h45m44.004582868s) [upload=726663]: Got next segment. Inline: false Jan 9 06:14:44.133570127 (830h45m44.004653162s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:14:44.133577421 (830h45m44.004660456s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:14:44.376950314 (830h45m44.248033349s) [seg_pos={0 0}, upload=726663]: Begin upload segment... Jan 9 06:14:44.377003566 (830h45m44.248086601s) [seg_pos={0 0}, upload=726663]: Done begin upload segment. Jan 9 06:14:44.377040186 (830h45m44.248123231s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377085161 (830h45m44.248168206s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377548564 (830h45m44.248631609s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377722315 (830h45m44.24880536s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378124310 (830h45m44.249207355s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378312648 (830h45m44.249395683s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378696029 (830h45m44.249779064s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379067125 (830h45m44.25015016s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379131979 (830h45m44.250215024s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379615760 (830h45m44.250698805s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379960707 (830h45m44.251043742s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380240650 (830h45m44.251323695s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380602809 (830h45m44.251685844s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380827668 (830h45m44.251910703s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381186972 (830h45m44.252270007s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381350423 (830h45m44.252433459s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:46.084711067 (830h45m45.955794102s) [upload=726663]: Next returned nil segment Jan 9 06:14:46.084724584 (830h45m45.955807619s) [upload=726663]: Waiting for error group managing segments... Jan 9 06:14:46.371445789 (830h45m46.242528824s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.372939333 (830h45m46.244022378s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385195200 (830h45m46.256278235s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385700692 (830h45m46.256783737s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.390346538 (830h45m46.261429573s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.398302218 (830h45m46.269385263s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.418777034 (830h45m46.289860069s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.445630346 (830h45m46.316713381s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.554868320 (830h45m46.425951355s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.735038076 (830h45m46.606121121s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.752429242 (830h45m46.623512287s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.777515640 (830h45m46.648598685s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780170044 (830h45m46.651253089s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780182227 (830h45m46.651265272s) [seg_pos={0 0}, upload=726663]: Segment reached optimal threshold of 13 pieces. Jan 9 06:14:46.780328837 (830h45m46.651411872s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780336381 (830h45m46.651419416s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780487759 (830h45m46.651570804s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780498510 (830h45m46.651581545s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780537825 (830h45m46.65162086s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection Jan 9 06:14:46.780544738 (830h45m46.651627773s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780559826 (830h45m46.651642861s) [seg_pos={0 0}, upload=726663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:14:46.780622375 (830h45m46.65170541s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:14:46.780626643 (830h45m46.651709688s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:14:47.082429286 (830h45m46.953512321s) [upload=726663]: Done uploading segments... Jan 9 06:14:47.082454053 (830h45m46.953537098s) [upload=726663]: Upload finished. err: <nil> Duration: 29.697589879s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000911591s trace: https://jaeger.dev.storj.io/trace/7e0422fd5b50fdb FAIL (2026-01-09 06:22:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000934485s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 18.346032031s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:22:38 UTC) orbiter warning: context deadline exceeded Duration: 18.346048381s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........XX....X...........X........X.X...XX.X..X..........] duration: 6.089074707s trace: https://jaeger.dev.storj.io/trace/767dc81c2be0328 FAIL (2026-01-09 06:22:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ) context deadline exceeded; piecestore: (Node ID: 1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, Piece ID: OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A) context deadline exceeded ======LOGS====== 9 06:14:44.133209750 (830h45m44.004292795s) [upload=726663]: Starting upload Jan 9 06:14:44.133266649 (830h45m44.004349684s) [upload=726663]: Done starting upload Jan 9 06:14:44.133397809 (830h45m44.004480844s) [upload=726663]: Uploading segments... Jan 9 06:14:44.133499823 (830h45m44.004582868s) [upload=726663]: Got next segment. Inline: false Jan 9 06:14:44.133570127 (830h45m44.004653162s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:14:44.133577421 (830h45m44.004660456s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:14:44.376950314 (830h45m44.248033349s) [seg_pos={0 0}, upload=726663]: Begin upload segment... Jan 9 06:14:44.377003566 (830h45m44.248086601s) [seg_pos={0 0}, upload=726663]: Done begin upload segment. Jan 9 06:14:44.377040186 (830h45m44.248123231s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377085161 (830h45m44.248168206s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377548564 (830h45m44.248631609s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377722315 (830h45m44.24880536s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378124310 (830h45m44.249207355s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378312648 (830h45m44.249395683s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378696029 (830h45m44.249779064s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379067125 (830h45m44.25015016s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379131979 (830h45m44.250215024s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379615760 (830h45m44.250698805s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379960707 (830h45m44.251043742s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380240650 (830h45m44.251323695s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380602809 (830h45m44.251685844s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380827668 (830h45m44.251910703s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381186972 (830h45m44.252270007s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381350423 (830h45m44.252433459s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:46.084711067 (830h45m45.955794102s) [upload=726663]: Next returned nil segment Jan 9 06:14:46.084724584 (830h45m45.955807619s) [upload=726663]: Waiting for error group managing segments... Jan 9 06:14:46.371445789 (830h45m46.242528824s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.372939333 (830h45m46.244022378s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385195200 (830h45m46.256278235s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385700692 (830h45m46.256783737s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.390346538 (830h45m46.261429573s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.398302218 (830h45m46.269385263s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.418777034 (830h45m46.289860069s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.445630346 (830h45m46.316713381s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.554868320 (830h45m46.425951355s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.735038076 (830h45m46.606121121s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.752429242 (830h45m46.623512287s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.777515640 (830h45m46.648598685s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780170044 (830h45m46.651253089s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780182227 (830h45m46.651265272s) [seg_pos={0 0}, upload=726663]: Segment reached optimal threshold of 13 pieces. Jan 9 06:14:46.780328837 (830h45m46.651411872s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780336381 (830h45m46.651419416s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780487759 (830h45m46.651570804s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780498510 (830h45m46.651581545s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780537825 (830h45m46.65162086s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection Jan 9 06:14:46.780544738 (830h45m46.651627773s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780559826 (830h45m46.651642861s) [seg_pos={0 0}, upload=726663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:14:46.780622375 (830h45m46.65170541s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:14:46.780626643 (830h45m46.651709688s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:14:47.082429286 (830h45m46.953512321s) [upload=726663]: Done uploading segments... Jan 9 06:14:47.082454053 (830h45m46.953537098s) [upload=726663]: Upload finished. err: <nil> Duration: 6.089099694s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 20.99417114s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:22:20 UTC) orbiter warning: context deadline exceeded Duration: 20.994194355s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [X.XXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001581905s trace: https://jaeger.dev.storj.io/trace/6f5ed270c44d977c FAIL (2026-01-09 06:21:49 UTC) orbiter warning: context deadline exceeded Duration: 5.001603537s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 22.425511007s trace: https://jaeger.dev.storj.io/trace/1d17c7c9f08684f5 FAIL (2026-01-09 06:21:46 UTC) orbiter warning: context deadline exceeded Duration: 22.425533732s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...........XX....X...........X........X.X...XX.X..X.........] duration: 6.171738179s trace: https://jaeger.dev.storj.io/trace/767dc81c2be0328 FAIL (2026-01-09 06:21:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ) context deadline exceeded; piecestore: (Node ID: 1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, Piece ID: OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A) context deadline exceeded ======LOGS====== 9 06:14:44.133209750 (830h45m44.004292795s) [upload=726663]: Starting upload Jan 9 06:14:44.133266649 (830h45m44.004349684s) [upload=726663]: Done starting upload Jan 9 06:14:44.133397809 (830h45m44.004480844s) [upload=726663]: Uploading segments... Jan 9 06:14:44.133499823 (830h45m44.004582868s) [upload=726663]: Got next segment. Inline: false Jan 9 06:14:44.133570127 (830h45m44.004653162s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:14:44.133577421 (830h45m44.004660456s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:14:44.376950314 (830h45m44.248033349s) [seg_pos={0 0}, upload=726663]: Begin upload segment... Jan 9 06:14:44.377003566 (830h45m44.248086601s) [seg_pos={0 0}, upload=726663]: Done begin upload segment. Jan 9 06:14:44.377040186 (830h45m44.248123231s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377085161 (830h45m44.248168206s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377548564 (830h45m44.248631609s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.377722315 (830h45m44.24880536s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378124310 (830h45m44.249207355s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378312648 (830h45m44.249395683s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.378696029 (830h45m44.249779064s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379067125 (830h45m44.25015016s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379131979 (830h45m44.250215024s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379615760 (830h45m44.250698805s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.379960707 (830h45m44.251043742s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380240650 (830h45m44.251323695s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380602809 (830h45m44.251685844s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.380827668 (830h45m44.251910703s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381186972 (830h45m44.252270007s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:44.381350423 (830h45m44.252433459s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Uploading piece... Jan 9 06:14:46.084711067 (830h45m45.955794102s) [upload=726663]: Next returned nil segment Jan 9 06:14:46.084724584 (830h45m45.955807619s) [upload=726663]: Waiting for error group managing segments... Jan 9 06:14:46.371445789 (830h45m46.242528824s) [noise=true , address=145.79.227.40:4051 , piece_id=EX7L7YFMYVFVOD7HXLGYREYPKYO65PJD2TR3WNPW336KFGU4FG2A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.372939333 (830h45m46.244022378s) [noise=true , address=145.79.227.46:4601 , piece_id=QNSFDJC7AQ6673AWTSZKY5UL3D27LVY5GORBAKTCOQWC7C3VTOXA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385195200 (830h45m46.256278235s) [noise=true , address=145.79.227.48:4303 , piece_id=OCWVSWRKTUAXAXIIRKMUNBP3TMKZRWZXQ75A5PR7F7P7ZAGOKHVQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.385700692 (830h45m46.256783737s) [noise=true , address=145.79.227.5:4223 , piece_id=HKNTX2GPGGLV3OJSJX4SF7CUZJEXNBC6UG7H5SRS32KA6XKF7WOA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.390346538 (830h45m46.261429573s) [noise=true , address=145.79.227.23:4545 , piece_id=DWVKREBQV3DHLPMYGHP42TC7Q7SIBUYXK47K7RMP4KME5LAZFW7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.398302218 (830h45m46.269385263s) [noise=true , address=145.79.227.6:4258 , piece_id=XMBTQWFHLLQA7HGVKNFXUBSHGUSTKONDEBDX3BPZMTBRTQFG6OXQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.418777034 (830h45m46.289860069s) [noise=true , address=145.79.227.44:4193 , piece_id=6RBDB2YZHV5DBUA2NNT7JVSK3DUJWHSEVEBJILEFD7QLSRXGHVYQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.445630346 (830h45m46.316713381s) [noise=true , address=145.79.227.50:4375 , piece_id=AAV6EDI54SAZD4EAKVWINWHWNSS2LIPADOPF5YI2ENCHJYZEX4PQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.554868320 (830h45m46.425951355s) [noise=true , address=145.79.227.41:4085 , piece_id=6EJRRATJ2JIPQL3MBL63LLGIMJKEU2ZFE2KZZ4J3IN5USFD2VXMQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.735038076 (830h45m46.606121121s) [noise=true , address=77.90.31.12:4589 , piece_id=OJBODWX7P42G2JV4XDNVLWABJV65DTWYSN56CBSGSKDLZJXUQG3A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.752429242 (830h45m46.623512287s) [noise=true , address=45.91.116.11:4119 , piece_id=CNNARZRIRCWAEUPIKBUHUM5GPHLYB7S6IT3WJKX5DMGCM4BDPC7Q, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.777515640 (830h45m46.648598685s) [noise=true , address=45.91.116.4:4940 , piece_id=RDWEILV4CRLQQLQCVTDFHZVNGU3R4OBPQ2GTZBQ73U7MRJAR6FCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780170044 (830h45m46.651253089s) [noise=true , address=45.91.116.13:4154 , piece_id=WLC3GPXZ7BQ3NHQEQFE6L62BTAT4OJJZXI66FUAFUN2PYHT3ASUQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: <nil> Jan 9 06:14:46.780182227 (830h45m46.651265272s) [seg_pos={0 0}, upload=726663]: Segment reached optimal threshold of 13 pieces. Jan 9 06:14:46.780328837 (830h45m46.651411872s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780336381 (830h45m46.651419416s) [noise=true , address=77.90.31.17:4770 , piece_id=RKRLXGGUZ4BYQXSRBK2VB3P7IXJC3J7S2EBECWKOVEL77IN7YLCQ, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780487759 (830h45m46.651570804s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jan 9 06:14:46.780498510 (830h45m46.651581545s) [noise=true , address=45.91.116.5:4299 , piece_id=PCBBNMMNMYWX4XZ2252IMP5QII3LZ777GNBOJPRIYMNDM2VCV46A, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780537825 (830h45m46.65162086s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59572->77.90.31.18:4805: use of closed network connection Jan 9 06:14:46.780544738 (830h45m46.651627773s) [noise=true , address=77.90.31.18:4805 , piece_id=DEOZOWTLXMIFSTBA74EWVXN7SQLDZVKFA5XB572VHGLCY4UOWVCA, seg_pos={0 0}, upload=726663]: Long tail cancellation due to optimal threshold Jan 9 06:14:46.780559826 (830h45m46.651642861s) [seg_pos={0 0}, upload=726663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:14:46.780622375 (830h45m46.65170541s) [upload=726663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:14:46.780626643 (830h45m46.651709688s) [upload=726663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:14:47.082429286 (830h45m46.953512321s) [upload=726663]: Done uploading segments... Jan 9 06:14:47.082454053 (830h45m46.953537098s) [upload=726663]: Upload finished. err: <nil> Duration: 6.171761774s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)