Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X.X.....................X........X....] duration: 19.805042538s trace: https://jaeger.dev.storj.io/trace/7b688950b65ff09b FAIL (2025-12-23 13:30:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: 5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ) context deadline exceeded ======LOGS====== Dec 23 13:27:50.028281060 (429h58m49.899364095s) [upload=377468]: Starting upload Dec 23 13:27:50.028349740 (429h58m49.899432775s) [upload=377468]: Done starting upload Dec 23 13:27:50.028413512 (429h58m49.899496557s) [upload=377468]: Uploading segments... Dec 23 13:27:50.028578536 (429h58m49.899661582s) [upload=377468]: Got next segment. Inline: false Dec 23 13:27:50.028610287 (429h58m49.899693322s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:27:50.028615046 (429h58m49.899698081s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:27:50.270611004 (429h58m50.141694059s) [seg_pos={0 0}, upload=377468]: Begin upload segment... Dec 23 13:27:50.270681819 (429h58m50.141764854s) [seg_pos={0 0}, upload=377468]: Done begin upload segment. Dec 23 13:27:50.270742866 (429h58m50.141825901s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.270918841 (429h58m50.142001876s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.271591561 (429h58m50.142674596s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272183318 (429h58m50.143266363s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272619960 (429h58m50.143702995s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273140702 (429h58m50.144223747s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273691290 (429h58m50.144774325s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274157948 (429h58m50.145240993s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274584110 (429h58m50.145667155s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275006103 (429h58m50.146089168s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275392860 (429h58m50.146475895s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275475217 (429h58m50.146558262s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275779396 (429h58m50.146862441s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276142157 (429h58m50.147225192s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276533021 (429h58m50.147616056s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276609086 (429h58m50.147692121s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:51.963189226 (429h58m51.834272261s) [upload=377468]: Next returned nil segment Dec 23 13:27:51.963217350 (429h58m51.834300385s) [upload=377468]: Waiting for error group managing segments... Dec 23 13:27:52.243617666 (429h58m52.114700702s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.255799703 (429h58m52.126882738s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.258185617 (429h58m52.129268662s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.260409782 (429h58m52.131492817s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.263338509 (429h58m52.134421544s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.268551075 (429h58m52.13963412s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.275110525 (429h58m52.14619356s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281384641 (429h58m52.152467686s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281920401 (429h58m52.153003436s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.283881566 (429h58m52.154964601s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.297894188 (429h58m52.168977223s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.507803499 (429h58m52.378886544s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704106441 (429h58m52.575189476s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704126620 (429h58m52.575209655s) [seg_pos={0 0}, upload=377468]: Segment reached optimal threshold of 13 pieces. Dec 23 13:27:52.704410060 (429h58m52.575493095s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection Dec 23 13:27:52.704416792 (429h58m52.575499847s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704540669 (429h58m52.575623704s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Dec 23 13:27:52.704546209 (429h58m52.575629254s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704690754 (429h58m52.575773799s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704756329 (429h58m52.575839374s) [seg_pos={0 0}, upload=377468]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 23 13:27:52.704898400 (429h58m52.575981445s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:27:52.704908069 (429h58m52.575991114s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:27:52.996134099 (429h58m52.867217144s) [upload=377468]: Done uploading segments... Dec 23 13:27:52.996154438 (429h58m52.867237484s) [upload=377468]: Upload finished. err: <nil> Duration: 19.805058918s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 252.243203ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:30:20 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.269132ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................X.X.....................X........X...] duration: 5.945645796s trace: https://jaeger.dev.storj.io/trace/7b688950b65ff09b FAIL (2025-12-23 13:29:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: 5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ) context deadline exceeded ======LOGS====== Dec 23 13:27:50.028281060 (429h58m49.899364095s) [upload=377468]: Starting upload Dec 23 13:27:50.028349740 (429h58m49.899432775s) [upload=377468]: Done starting upload Dec 23 13:27:50.028413512 (429h58m49.899496557s) [upload=377468]: Uploading segments... Dec 23 13:27:50.028578536 (429h58m49.899661582s) [upload=377468]: Got next segment. Inline: false Dec 23 13:27:50.028610287 (429h58m49.899693322s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:27:50.028615046 (429h58m49.899698081s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:27:50.270611004 (429h58m50.141694059s) [seg_pos={0 0}, upload=377468]: Begin upload segment... Dec 23 13:27:50.270681819 (429h58m50.141764854s) [seg_pos={0 0}, upload=377468]: Done begin upload segment. Dec 23 13:27:50.270742866 (429h58m50.141825901s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.270918841 (429h58m50.142001876s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.271591561 (429h58m50.142674596s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272183318 (429h58m50.143266363s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272619960 (429h58m50.143702995s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273140702 (429h58m50.144223747s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273691290 (429h58m50.144774325s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274157948 (429h58m50.145240993s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274584110 (429h58m50.145667155s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275006103 (429h58m50.146089168s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275392860 (429h58m50.146475895s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275475217 (429h58m50.146558262s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275779396 (429h58m50.146862441s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276142157 (429h58m50.147225192s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276533021 (429h58m50.147616056s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276609086 (429h58m50.147692121s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:51.963189226 (429h58m51.834272261s) [upload=377468]: Next returned nil segment Dec 23 13:27:51.963217350 (429h58m51.834300385s) [upload=377468]: Waiting for error group managing segments... Dec 23 13:27:52.243617666 (429h58m52.114700702s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.255799703 (429h58m52.126882738s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.258185617 (429h58m52.129268662s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.260409782 (429h58m52.131492817s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.263338509 (429h58m52.134421544s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.268551075 (429h58m52.13963412s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.275110525 (429h58m52.14619356s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281384641 (429h58m52.152467686s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281920401 (429h58m52.153003436s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.283881566 (429h58m52.154964601s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.297894188 (429h58m52.168977223s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.507803499 (429h58m52.378886544s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704106441 (429h58m52.575189476s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704126620 (429h58m52.575209655s) [seg_pos={0 0}, upload=377468]: Segment reached optimal threshold of 13 pieces. Dec 23 13:27:52.704410060 (429h58m52.575493095s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection Dec 23 13:27:52.704416792 (429h58m52.575499847s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704540669 (429h58m52.575623704s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Dec 23 13:27:52.704546209 (429h58m52.575629254s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704690754 (429h58m52.575773799s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704756329 (429h58m52.575839374s) [seg_pos={0 0}, upload=377468]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 23 13:27:52.704898400 (429h58m52.575981445s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:27:52.704908069 (429h58m52.575991114s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:27:52.996134099 (429h58m52.867217144s) [upload=377468]: Done uploading segments... Dec 23 13:27:52.996154438 (429h58m52.867237484s) [upload=377468]: Upload finished. err: <nil> Duration: 5.945675664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................X.X.....................X........X..] duration: 5.708255357s trace: https://jaeger.dev.storj.io/trace/7b688950b65ff09b FAIL (2025-12-23 13:29:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: 5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ) context deadline exceeded ======LOGS====== Dec 23 13:27:50.028281060 (429h58m49.899364095s) [upload=377468]: Starting upload Dec 23 13:27:50.028349740 (429h58m49.899432775s) [upload=377468]: Done starting upload Dec 23 13:27:50.028413512 (429h58m49.899496557s) [upload=377468]: Uploading segments... Dec 23 13:27:50.028578536 (429h58m49.899661582s) [upload=377468]: Got next segment. Inline: false Dec 23 13:27:50.028610287 (429h58m49.899693322s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:27:50.028615046 (429h58m49.899698081s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:27:50.270611004 (429h58m50.141694059s) [seg_pos={0 0}, upload=377468]: Begin upload segment... Dec 23 13:27:50.270681819 (429h58m50.141764854s) [seg_pos={0 0}, upload=377468]: Done begin upload segment. Dec 23 13:27:50.270742866 (429h58m50.141825901s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.270918841 (429h58m50.142001876s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.271591561 (429h58m50.142674596s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272183318 (429h58m50.143266363s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272619960 (429h58m50.143702995s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273140702 (429h58m50.144223747s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273691290 (429h58m50.144774325s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274157948 (429h58m50.145240993s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274584110 (429h58m50.145667155s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275006103 (429h58m50.146089168s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275392860 (429h58m50.146475895s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275475217 (429h58m50.146558262s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275779396 (429h58m50.146862441s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276142157 (429h58m50.147225192s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276533021 (429h58m50.147616056s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276609086 (429h58m50.147692121s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:51.963189226 (429h58m51.834272261s) [upload=377468]: Next returned nil segment Dec 23 13:27:51.963217350 (429h58m51.834300385s) [upload=377468]: Waiting for error group managing segments... Dec 23 13:27:52.243617666 (429h58m52.114700702s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.255799703 (429h58m52.126882738s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.258185617 (429h58m52.129268662s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.260409782 (429h58m52.131492817s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.263338509 (429h58m52.134421544s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.268551075 (429h58m52.13963412s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.275110525 (429h58m52.14619356s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281384641 (429h58m52.152467686s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281920401 (429h58m52.153003436s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.283881566 (429h58m52.154964601s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.297894188 (429h58m52.168977223s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.507803499 (429h58m52.378886544s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704106441 (429h58m52.575189476s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704126620 (429h58m52.575209655s) [seg_pos={0 0}, upload=377468]: Segment reached optimal threshold of 13 pieces. Dec 23 13:27:52.704410060 (429h58m52.575493095s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection Dec 23 13:27:52.704416792 (429h58m52.575499847s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704540669 (429h58m52.575623704s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Dec 23 13:27:52.704546209 (429h58m52.575629254s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704690754 (429h58m52.575773799s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704756329 (429h58m52.575839374s) [seg_pos={0 0}, upload=377468]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 23 13:27:52.704898400 (429h58m52.575981445s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:27:52.704908069 (429h58m52.575991114s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:27:52.996134099 (429h58m52.867217144s) [upload=377468]: Done uploading segments... Dec 23 13:27:52.996154438 (429h58m52.867237484s) [upload=377468]: Upload finished. err: <nil> Duration: 5.708280745s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 253.087129ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:29:09 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 253.113418ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X.X.....................X........X.] duration: 5.648818236s trace: https://jaeger.dev.storj.io/trace/7b688950b65ff09b FAIL (2025-12-23 13:28:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: 5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ) context deadline exceeded ======LOGS====== Dec 23 13:27:50.028281060 (429h58m49.899364095s) [upload=377468]: Starting upload Dec 23 13:27:50.028349740 (429h58m49.899432775s) [upload=377468]: Done starting upload Dec 23 13:27:50.028413512 (429h58m49.899496557s) [upload=377468]: Uploading segments... Dec 23 13:27:50.028578536 (429h58m49.899661582s) [upload=377468]: Got next segment. Inline: false Dec 23 13:27:50.028610287 (429h58m49.899693322s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:27:50.028615046 (429h58m49.899698081s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:27:50.270611004 (429h58m50.141694059s) [seg_pos={0 0}, upload=377468]: Begin upload segment... Dec 23 13:27:50.270681819 (429h58m50.141764854s) [seg_pos={0 0}, upload=377468]: Done begin upload segment. Dec 23 13:27:50.270742866 (429h58m50.141825901s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.270918841 (429h58m50.142001876s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.271591561 (429h58m50.142674596s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272183318 (429h58m50.143266363s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272619960 (429h58m50.143702995s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273140702 (429h58m50.144223747s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273691290 (429h58m50.144774325s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274157948 (429h58m50.145240993s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274584110 (429h58m50.145667155s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275006103 (429h58m50.146089168s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275392860 (429h58m50.146475895s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275475217 (429h58m50.146558262s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275779396 (429h58m50.146862441s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276142157 (429h58m50.147225192s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276533021 (429h58m50.147616056s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276609086 (429h58m50.147692121s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:51.963189226 (429h58m51.834272261s) [upload=377468]: Next returned nil segment Dec 23 13:27:51.963217350 (429h58m51.834300385s) [upload=377468]: Waiting for error group managing segments... Dec 23 13:27:52.243617666 (429h58m52.114700702s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.255799703 (429h58m52.126882738s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.258185617 (429h58m52.129268662s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.260409782 (429h58m52.131492817s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.263338509 (429h58m52.134421544s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.268551075 (429h58m52.13963412s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.275110525 (429h58m52.14619356s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281384641 (429h58m52.152467686s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281920401 (429h58m52.153003436s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.283881566 (429h58m52.154964601s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.297894188 (429h58m52.168977223s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.507803499 (429h58m52.378886544s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704106441 (429h58m52.575189476s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704126620 (429h58m52.575209655s) [seg_pos={0 0}, upload=377468]: Segment reached optimal threshold of 13 pieces. Dec 23 13:27:52.704410060 (429h58m52.575493095s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection Dec 23 13:27:52.704416792 (429h58m52.575499847s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704540669 (429h58m52.575623704s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Dec 23 13:27:52.704546209 (429h58m52.575629254s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704690754 (429h58m52.575773799s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704756329 (429h58m52.575839374s) [seg_pos={0 0}, upload=377468]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 23 13:27:52.704898400 (429h58m52.575981445s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:27:52.704908069 (429h58m52.575991114s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:27:52.996134099 (429h58m52.867217144s) [upload=377468]: Done uploading segments... Dec 23 13:27:52.996154438 (429h58m52.867237484s) [upload=377468]: Upload finished. err: <nil> Duration: 5.648844197s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 274.455038ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:28:44 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 274.487769ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................X.X.....................X........X] duration: 30.002059724s trace: https://jaeger.dev.storj.io/trace/7b688950b65ff09b FAIL (2025-12-23 13:28:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: 5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ) context deadline exceeded ======LOGS====== Dec 23 13:27:50.028281060 (429h58m49.899364095s) [upload=377468]: Starting upload Dec 23 13:27:50.028349740 (429h58m49.899432775s) [upload=377468]: Done starting upload Dec 23 13:27:50.028413512 (429h58m49.899496557s) [upload=377468]: Uploading segments... Dec 23 13:27:50.028578536 (429h58m49.899661582s) [upload=377468]: Got next segment. Inline: false Dec 23 13:27:50.028610287 (429h58m49.899693322s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:27:50.028615046 (429h58m49.899698081s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:27:50.270611004 (429h58m50.141694059s) [seg_pos={0 0}, upload=377468]: Begin upload segment... Dec 23 13:27:50.270681819 (429h58m50.141764854s) [seg_pos={0 0}, upload=377468]: Done begin upload segment. Dec 23 13:27:50.270742866 (429h58m50.141825901s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.270918841 (429h58m50.142001876s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.271591561 (429h58m50.142674596s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272183318 (429h58m50.143266363s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.272619960 (429h58m50.143702995s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273140702 (429h58m50.144223747s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.273691290 (429h58m50.144774325s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274157948 (429h58m50.145240993s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.274584110 (429h58m50.145667155s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275006103 (429h58m50.146089168s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275392860 (429h58m50.146475895s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275475217 (429h58m50.146558262s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.275779396 (429h58m50.146862441s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276142157 (429h58m50.147225192s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276533021 (429h58m50.147616056s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:50.276609086 (429h58m50.147692121s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Uploading piece... Dec 23 13:27:51.963189226 (429h58m51.834272261s) [upload=377468]: Next returned nil segment Dec 23 13:27:51.963217350 (429h58m51.834300385s) [upload=377468]: Waiting for error group managing segments... Dec 23 13:27:52.243617666 (429h58m52.114700702s) [noise=true , address=145.79.227.7:4293 , piece_id=OE23VLD4KTPN6ZTRDQQEARZNQN3265MC777ICQ3K632PXLFK35MQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.255799703 (429h58m52.126882738s) [noise=true , address=145.79.227.42:4122 , piece_id=ZNYBZ3Z3ERSSTE7O653Q5DCZLX3HW2BUZID3EGPJQM7C7ELIKHFA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.258185617 (429h58m52.129268662s) [noise=true , address=145.79.227.48:4302 , piece_id=TZ5YDXEMLSQGFFMRW7ACYETVUEL5QCABG37TG4Q6HUS7WZH5Q6UA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.260409782 (429h58m52.131492817s) [noise=true , address=145.79.227.51:4410 , piece_id=WKPYZ42JMENMHSNEYUULRZWYVTAR5TLA4E5N22HS6BRQDRIPUJ4Q, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.263338509 (429h58m52.134421544s) [noise=true , address=145.79.227.41:4085 , piece_id=MK6BTOL4GOTURGWMBPK26VX4CP6E6WLTGZTLIDXFHG3GAHTJKPMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.268551075 (429h58m52.13963412s) [noise=true , address=145.79.227.35:4482 , piece_id=AIAAA3KZ2DWJNOT22VKQAND33V7REC6B4NA75U4SYM6DINGWXR6A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.275110525 (429h58m52.14619356s) [noise=true , address=145.79.227.21:4474 , piece_id=XCWX72V7RTYAJKQSGNCRGTVJGUC5TMMDN2FEPY3WZH5M72G37PYQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281384641 (429h58m52.152467686s) [noise=true , address=145.79.227.43:4157 , piece_id=R37VDET7UBREYM4XMGJPEGBTUXJWSSP4QXIBNIUID2RCBYGHVPAA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.281920401 (429h58m52.153003436s) [noise=true , address=145.79.227.14:4839 , piece_id=OVOHG4KGONSOVUCNKZC6TB4GEMSWF56RNPWYGXE4H5G6H2CFRWMQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.283881566 (429h58m52.154964601s) [noise=true , address=145.79.227.46:4602 , piece_id=ZWZ77UCM37Z6NFPT32AT7D4IRQNZ6FM4CKEPX4Z7XDO5P5Y7IJBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.297894188 (429h58m52.168977223s) [noise=true , address=145.79.227.20:4438 , piece_id=LKUYUWSXV7PVCITWIOTJJJWUGM7ZJALJTBXDXMUKVKWGAWM3SP3A, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.507803499 (429h58m52.378886544s) [noise=true , address=45.91.116.8:4048 , piece_id=K7AJ26PRDLOCKJIH7WNESKC3V7PDDVKVXTMO3MTYA3RHOMRZSNIA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704106441 (429h58m52.575189476s) [noise=true , address=45.91.116.11:4120 , piece_id=Q6LLDYKH5Q7PQQWIV7GEO5QWU2OHHWD4FNEQPBXY7KM3XQ456FDQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704126620 (429h58m52.575209655s) [seg_pos={0 0}, upload=377468]: Segment reached optimal threshold of 13 pieces. Dec 23 13:27:52.704410060 (429h58m52.575493095s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50874->77.90.31.10:4517: use of closed network connection Dec 23 13:27:52.704416792 (429h58m52.575499847s) [noise=true , address=77.90.31.10:4517 , piece_id=ZZER7LEHPTGAU37ZVCINZOM776RBZAZUHDEVMGUGWTCZ2VXE5BNA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704540669 (429h58m52.575623704s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Dec 23 13:27:52.704546209 (429h58m52.575629254s) [noise=true , address=145.79.227.40:4051 , piece_id=EOQJCESKRDQ3MVFN2OHAG322NG7N3KWASEL7R2KL2AMNLNSCZUUA, seg_pos={0 0}, upload=377468]: Long tail cancellation due to optimal threshold Dec 23 13:27:52.704690754 (429h58m52.575773799s) [noise=true , address=45.91.116.14:4191 , piece_id=5NG2NRIN6BP73YTEL46V3RWGOLPZWKVDBWQ63WOBXBMUN7LURHBQ, seg_pos={0 0}, upload=377468]: Done uploading piece. err: <nil> Dec 23 13:27:52.704756329 (429h58m52.575839374s) [seg_pos={0 0}, upload=377468]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 23 13:27:52.704898400 (429h58m52.575981445s) [upload=377468]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:27:52.704908069 (429h58m52.575991114s) [upload=377468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:27:52.996134099 (429h58m52.867217144s) [upload=377468]: Done uploading segments... Dec 23 13:27:52.996154438 (429h58m52.867237484s) [upload=377468]: Upload finished. err: <nil> Duration: 30.002073349s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 263.819335ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:27:58 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 263.853181ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 270.337316ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:27:22 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 270.366472ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.X.....................X........] duration: 5.723468833s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:27:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 5.723501705s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 248.305602ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:26:55 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 248.330971ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 260.165337ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:26:09 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 260.193621ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X.X.....................X.......] duration: 5.953043378s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:25:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 5.953069758s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 264.226198ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:25:04 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 264.261517ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.X.....................X......] duration: 6.065927252s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:24:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 6.066038443s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 242.073446ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:24:14 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 242.099307ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.X.....................X.....] duration: 5.634182479s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:24:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 5.634209782s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 248.509233ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:23:45 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 248.542755ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.X.....................X....] duration: 5.626306353s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:23:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 5.62635708s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 943.579509ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:22:36 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 943.600669ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X.X.....................X...] duration: 6.033260769s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:22:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 6.033284794s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 249.620758ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:22:06 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 249.652799ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................X.X.....................X..] duration: 6.281736084s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:21:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 6.281760971s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 259.067496ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:20:53 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 259.092042ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................X.X.....................X.] duration: 5.959786783s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:20:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 5.959819766s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 248.393992ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:20:21 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 248.423287ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................X.X.....................X] duration: 30.004700727s trace: https://jaeger.dev.storj.io/trace/3e2986326adc501c FAIL (2025-12-23 13:20:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ) context deadline exceeded ======LOGS====== nts... Dec 23 13:19:36.392854796 (429h50m36.263937831s) [upload=377359]: Got next segment. Inline: false Dec 23 13:19:36.392904981 (429h50m36.263988026s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:19:36.392911473 (429h50m36.263994518s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:19:36.640017563 (429h50m36.511100598s) [seg_pos={0 0}, upload=377359]: Begin upload segment... Dec 23 13:19:36.640153101 (429h50m36.511236136s) [seg_pos={0 0}, upload=377359]: Done begin upload segment. Dec 23 13:19:36.640252210 (429h50m36.511335245s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640288920 (429h50m36.511371955s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.640895875 (429h50m36.51197892s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641201638 (429h50m36.512284683s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641325544 (429h50m36.512408579s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641560811 (429h50m36.512643856s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641736406 (429h50m36.512819451s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.641970061 (429h50m36.513053106s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642271685 (429h50m36.513354721s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642427532 (429h50m36.513510567s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642670534 (429h50m36.51375358s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.642763171 (429h50m36.513846206s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643206335 (429h50m36.51428938s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.643267792 (429h50m36.514350837s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.644227570 (429h50m36.515310615s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:36.645460687 (429h50m36.516543732s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Uploading piece... Dec 23 13:19:38.345728403 (429h50m38.216811448s) [upload=377359]: Next returned nil segment Dec 23 13:19:38.345754674 (429h50m38.216837719s) [upload=377359]: Waiting for error group managing segments... Dec 23 13:19:38.632653706 (429h50m38.503736741s) [noise=true , address=145.79.227.43:4159 , piece_id=A4FJUNG7OS4BVT2XBKFGQCT2RFIOYKJLXHBIVPSXGRGF4SZCW2OA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.637373143 (429h50m38.508456178s) [noise=true , address=145.79.227.14:4839 , piece_id=V7JXZKRDNXYPDICW7YV7PWGPQIIK5JVTBFU5UE5UJD3NV6FBKV3Q, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.646588550 (429h50m38.517671585s) [noise=true , address=145.79.227.47:4266 , piece_id=J7OHV3UVD66KVR3GPIUI4LVGICSF5FOQRUPDW45OQFRC22IASDXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.647149248 (429h50m38.518232293s) [noise=true , address=145.79.227.41:4086 , piece_id=IKUK3GVVOUPAGXRGIU6NAV54HIKRNQO3VVEBK2CY2DZWE2GX5RKA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.649590887 (429h50m38.520673922s) [noise=true , address=145.79.227.46:4601 , piece_id=ZILXDDRD6YV7MRVEPJF6UVMGKUS5ZEASAGGOXNOI5CDHIX6PLESA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.654592652 (429h50m38.525675687s) [noise=true , address=145.79.227.6:4259 , piece_id=3UWTKA4RPTDXZZIO52BBXT3BR7P5ERWV65HPHJMLXWOVCXNVROHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655037178 (429h50m38.526120213s) [noise=true , address=145.79.227.45:4231 , piece_id=MFOYTII4NK32L23J3H3HZ6QG2Y4EHFKSZDKCI6F4CI5LHZJRQMQA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.655131187 (429h50m38.526214222s) [noise=true , address=145.79.227.38:4978 , piece_id=H5T2NXVRZNPZ6ZBJLHHD2UDUBC4X4JEB2GEEUO3NCH6WVRKMXXAQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.656192498 (429h50m38.527275533s) [noise=true , address=145.79.227.4:4187 , piece_id=27VE5GJGF36RHQNFGEC2VYIAKBESGH6WWCGK5YUI6NA4I62PPWXA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.667271073 (429h50m38.538354108s) [noise=true , address=145.79.227.34:4905 , piece_id=W6CXWG6ZH7JIK2DS4HKSMX6BM664J73WR2AYX7S3HTOY22YEDPHQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.672608648 (429h50m38.543691693s) [noise=true , address=145.79.227.36:4518 , piece_id=REFJBCX6YMRNINB6ELO343B5JLHCPEAS4ZH3V4MKS4ZKOTOQR4VA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:38.840954455 (429h50m38.71203749s) [noise=true , address=77.90.31.16:4735 , piece_id=HBS7KBHMKMJQMBR5HVASGONAXYUFSEWUM6HQJFU6XV2AJ7XAGNIQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146003117 (429h50m39.017086152s) [noise=true , address=45.91.116.14:4190 , piece_id=XENALRK23SCIOUXYQXJK2TCGIY4YZH7BABGIE3PXVHK2WES3RAJQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: <nil> Dec 23 13:19:39.146021181 (429h50m39.017104226s) [seg_pos={0 0}, upload=377359]: Segment reached optimal threshold of 13 pieces. Dec 23 13:19:39.146228356 (429h50m39.017311401s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53404->45.91.116.10:4082: use of closed network connection Dec 23 13:19:39.146235970 (429h50m39.017319005s) [noise=true , address=45.91.116.10:4082 , piece_id=CZQB7MADUTVEQ5SYT7XNCRNF2THBJG3THVZJ2LSBLHNZNVWEEO2A, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146251941 (429h50m39.017334986s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47708->45.91.116.13:4155: use of closed network connection Dec 23 13:19:39.146260928 (429h50m39.017343963s) [noise=true , address=45.91.116.13:4155 , piece_id=IR2DY3WUMXVFLLMYWK54RMZLYRXAVENBASU6QWNB24NXWVGMAHJA, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146356560 (429h50m39.017439605s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:19:39.146362361 (429h50m39.017445396s) [noise=true , address=45.91.116.12:4226 , piece_id=NYDL63SQCLLDH7S3YDJT4ON45MYLUR5MLUSQSOUJQPF5JUKPZABQ, seg_pos={0 0}, upload=377359]: Long tail cancellation due to optimal threshold Dec 23 13:19:39.146433757 (429h50m39.017516792s) [seg_pos={0 0}, upload=377359]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:19:39.146498550 (429h50m39.017581586s) [upload=377359]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:19:39.146504071 (429h50m39.017587106s) [upload=377359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:19:39.440194775 (429h50m39.31127782s) [upload=377359]: Done uploading segments... Dec 23 13:19:39.440229301 (429h50m39.311312336s) [upload=377359]: Upload finished. err: <nil> Duration: 30.004724123s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 258.786432ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:19:23 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 258.817701ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 252.112653ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:19:08 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.138973ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.X.....................] duration: 5.801756063s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:18:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 5.80178004s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 252.586939ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:18:50 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.617276ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 251.977346ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:18:21 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.004898ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.X....................] duration: 6.042075573s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:17:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 6.042101773s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 263.118491ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:17:57 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 263.144339ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.X...................] duration: 5.775991314s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:16:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 5.776020009s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 252.284831ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:16:56 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.326681ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.X..................] duration: 5.70549563s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:16:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 5.705515388s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 251.197341ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:16:00 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 251.221115ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.X.................] duration: 5.762751895s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:15:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 5.762810857s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 258.778404ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:15:19 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 259.093123ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.X................] duration: 6.392787704s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:14:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 6.392815747s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 259.365713ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:14:10 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 259.389119ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.X...............] duration: 6.392469518s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:13:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 6.392499827s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 256.90027ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:13:49 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 256.930707ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.X..............] duration: 6.149332879s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:13:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 6.149364448s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 252.452892ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:13:03 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.477668ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.X.............] duration: 6.438544475s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:12:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 6.438569322s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.X............] duration: 25.49491013s trace: https://jaeger.dev.storj.io/trace/335d60b5e3cb0611 FAIL (2025-12-23 13:12:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== gments... Dec 23 13:03:41.146773219 (429h34m41.017856254s) [upload=377134]: Got next segment. Inline: false Dec 23 13:03:41.146815550 (429h34m41.017898585s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 23 13:03:41.146820509 (429h34m41.017903544s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 23 13:03:41.395177081 (429h34m41.266260127s) [seg_pos={0 0}, upload=377134]: Begin upload segment... Dec 23 13:03:41.395218049 (429h34m41.266301084s) [seg_pos={0 0}, upload=377134]: Done begin upload segment. Dec 23 13:03:41.395250882 (429h34m41.266333927s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395268705 (429h34m41.26635175s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395658838 (429h34m41.266741873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395663367 (429h34m41.266746413s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.395962717 (429h34m41.267045762s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396003695 (429h34m41.26708673s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396288427 (429h34m41.267371462s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396312643 (429h34m41.267395678s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396575905 (429h34m41.26765894s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396676416 (429h34m41.267759461s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.396865416 (429h34m41.267948451s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397008699 (429h34m41.268091734s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397165558 (429h34m41.268248593s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397266479 (429h34m41.268349514s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.397521615 (429h34m41.268604661s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:41.398961146 (429h34m41.270044191s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Uploading piece... Dec 23 13:03:43.108995170 (429h34m42.980078205s) [upload=377134]: Next returned nil segment Dec 23 13:03:43.109012081 (429h34m42.980095146s) [upload=377134]: Waiting for error group managing segments... Dec 23 13:03:43.392567427 (429h34m43.263650462s) [noise=true , address=145.79.227.37:4565 , piece_id=HBJ366WIOAHI6ADVVEEDQQWXUS2E5KPUSFEP2KL4XO2X5LOTFKYQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.402738524 (429h34m43.273821559s) [noise=true , address=145.79.227.11:4731 , piece_id=3N5KV6DMUF4QI2ZD3WB55AMP6X2YAFWCWP7VM4OBFPFB5UZEPGZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.403357101 (429h34m43.274440156s) [noise=true , address=145.79.227.38:4977 , piece_id=O6QO5Q3IFUTUCDOK7HXTJUYKTDJ5RBWDYB4NGEAMZOHDDR3IIC4A, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407110529 (429h34m43.278193564s) [noise=true , address=145.79.227.21:4473 , piece_id=7BR47RIYUFP5FWPABRZROLAAMKGHEPZPL3IDLGQ3XQNKXAMJ6URA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.407617795 (429h34m43.27870083s) [noise=true , address=145.79.227.29:4942 , piece_id=2HHIYW2ZIIAKDV4CZCO74BAVPK457IVGTE4EJVGANHF65D5W7PNA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.408029740 (429h34m43.279112775s) [noise=true , address=145.79.227.20:4438 , piece_id=TAOBXUAGOSBITQLEPKPRC7CUW6VMUM3JVR7VJ4Y4V4EFZ4YKQ7HQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.412549485 (429h34m43.28363253s) [noise=true , address=145.79.227.52:4446 , piece_id=PU54PSDQBMMYEONFTQT5IDII5K3PU4RAFZ5LMHOOM7YNAJS47OXA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.432973536 (429h34m43.304056571s) [noise=true , address=145.79.227.34:4905 , piece_id=D54AOI2BG6TPMWQSOJIQFFPQUQRMNYVFDY4PI7JK2NMFHJZAI23Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.445500129 (429h34m43.316583164s) [noise=true , address=145.79.227.32:4835 , piece_id=ULJPGLR3BQ6H2CGZHO7C5XO2EVN7Q7EY5L4NHRXZ3MQ6U7S64MVA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.456746244 (429h34m43.327829279s) [noise=true , address=145.79.227.12:4765 , piece_id=NOLXPWRRNQXRZ4OA2KMMX5M6FYJR2S3AOGPDIPVHXHQB5HN4GMZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.644894828 (429h34m43.515977873s) [noise=true , address=45.91.116.5:4297 , piece_id=TQXE3Q6QFA7F63DGJTQKNWEHIP2TOVRCPRVGCX6XC57BY7PJB3UQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.713352172 (429h34m43.584435217s) [noise=true , address=45.91.116.6:4976 , piece_id=Y72PN4WXYYPJ2ZNRBR2XA6QMHHTZ6OYOGNCWDBVLZKTO3I4S5KZQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731354943 (429h34m43.602437988s) [noise=true , address=45.91.116.14:4191 , piece_id=J4OQFX6XPN6GB6RJIVPBZUIBDDPLCWMM6GYGJGT7BAZAUM4DBNCA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: <nil> Dec 23 13:03:43.731377275 (429h34m43.602460311s) [seg_pos={0 0}, upload=377134]: Segment reached optimal threshold of 13 pieces. Dec 23 13:03:43.731577968 (429h34m43.602661013s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54770->77.90.31.13:4629: use of closed network connection Dec 23 13:03:43.731585332 (429h34m43.602668377s) [noise=true , address=77.90.31.13:4629 , piece_id=VX2CVI5FXKWL3W6ZFTRXG5RREY37EBUEQAHSQVPGBTJ63SC7XA4Q, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731671216 (429h34m43.602754261s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Dec 23 13:03:43.731682317 (429h34m43.602765362s) [noise=true , address=45.91.116.12:4226 , piece_id=BN76ETRXTTAEDWXZWR3A4KEOHHVKV4OFMG2GE6UOV7DIBE2WHIZA, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731838263 (429h34m43.602921308s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43872->45.91.116.11:4120: use of closed network connection Dec 23 13:03:43.731847792 (429h34m43.602930837s) [noise=true , address=45.91.116.11:4120 , piece_id=FODNYYBM7M4TD5KQZYKWGSQXULEIPEBPE6BTUPF5AZZUZSLQ7ACQ, seg_pos={0 0}, upload=377134]: Long tail cancellation due to optimal threshold Dec 23 13:03:43.731869032 (429h34m43.602952067s) [seg_pos={0 0}, upload=377134]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 23 13:03:43.731968902 (429h34m43.603051947s) [upload=377134]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 13:03:43.731975024 (429h34m43.603058069s) [upload=377134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 13:03:44.027725380 (429h34m43.898808415s) [upload=377134]: Done uploading segments... Dec 23 13:03:44.027744758 (429h34m43.898827793s) [upload=377134]: Upload finished. err: <nil> Duration: 25.494953472s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 271.150203ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/73921c1a3ca0437b FAIL (2025-12-23 13:11:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 271.181823ms edge linksharing bgp eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)