Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.770895997s FAIL (2025-07-01 20:32:13 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== p 188.166.225.170:38826->45.91.116.14:4190: use of closed network connection Jul 1 20:30:07.397683164 (203h19m42.168324499s) [seg_pos={0 0}, upload=162621]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 20:30:07.397793445 (203h19m42.16843478s) [upload=162621]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 20:30:07.397832890 (203h19m42.168474225s) [upload=162621]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 20:30:07.690755035 (203h19m42.46139637s) [upload=162621]: Done uploading segments... Jul 1 20:30:07.690777598 (203h19m42.461418933s) [upload=162621]: Upload finished. err: <nil> Jul 1 20:30:51.075923771 (203h20m25.846565096s) [upload=162628]: Starting upload Jul 1 20:30:51.075979769 (203h20m25.846621104s) [upload=162628]: Done starting upload Jul 1 20:30:51.076046185 (203h20m25.84668752s) [upload=162628]: Uploading segments... Jul 1 20:30:51.076156256 (203h20m25.846797591s) [upload=162628]: Got next segment. Inline: false Jul 1 20:30:51.076184720 (203h20m25.846826055s) [upload=162628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 20:30:51.076189279 (203h20m25.846830614s) [upload=162628]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 20:30:51.307264904 (203h20m26.077906239s) [seg_pos={0 0}, upload=162628]: Begin upload segment... Jul 1 20:30:51.307309248 (203h20m26.077950583s) [seg_pos={0 0}, upload=162628]: Done begin upload segment. Jul 1 20:30:51.307323446 (203h20m26.077964771s) [noise=true , address=45.91.116.8:4048 , piece_id=2G2OOQLJPTUDY5PZSHQ2MNAV7774DPK55SELX5STL5W35MXSFKNQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307341239 (203h20m26.077982574s) [noise=true , address=77.90.31.12:4589 , piece_id=OAQAOA63GE45RWO52GQFOO6FGD4M2AW6ALCPMKCB7T25UDBFOBIA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307377640 (203h20m26.078018965s) [noise=true , address=45.91.116.13:4154 , piece_id=CWIE4GHPWQCF7YHM2QG62RJMBQBNKNXRK2JI4G6OBTPKN4VV73BA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307671732 (203h20m26.078313057s) [noise=true , address=104.36.82.14:4221 , piece_id=JK73AQZVYWALWCBO7DORATYWJOIKWT2WBBHSZ26Q3TJLBYDTYO2Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307680439 (203h20m26.078321774s) [noise=true , address=104.36.82.7:4909 , piece_id=JNUIUO5MFV5I3L4CTK6IY2CMQDKTVCGPAR6VQHIKE2BKPIQQNGHA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307944213 (203h20m26.078585548s) [noise=true , address=45.91.116.5:5299 , piece_id=V6X7BVMX3IP23A65VFY7PIHGCJPGKLTZAYH4QVPPUFB34M3JUQYQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308197999 (203h20m26.078839334s) [noise=true , address=104.36.82.2:4731 , piece_id=JXQL75TODICFVUAIEE4TRGI2DUYVNC3ZR3RNEARTKQS56IIKZATQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308448899 (203h20m26.079090234s) [noise=true , address=45.91.116.4:4937 , piece_id=CMO4GLDKCZ7KQWHM5S4XX5QB2L3KAHUKRUYXWNE5OPEJ464X2NBA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308746398 (203h20m26.079387733s) [noise=true , address=77.90.31.17:4770 , piece_id=OFDOKUGNGNDMRFTF4HF3BHPZO3HSDDJZL7NC6G7PQSPKZRZVWGJA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308975577 (203h20m26.079616912s) [noise=true , address=45.91.116.13:4155 , piece_id=WN7BEIS5XO4K3BGAHLFDGBXUK52OHWYBPFFH6GCNADP6HWYBM73Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309167925 (203h20m26.07980926s) [noise=true , address=45.91.116.9:5262 , piece_id=N2SGOATWGVAKHJUCGXIUCMO5GTHQS4FHSFXZJBSGGT4HIQK4MFSQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309359802 (203h20m26.080001137s) [noise=true , address=45.91.116.6:4975 , piece_id=YUJIO2VMD53TDD66ANUI55LFMDTPHRMKV4Y7L7X66KA2BMKTIUQA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309548883 (203h20m26.080190218s) [noise=true , address=45.91.116.10:4082 , piece_id=U3OGQJQN3KWNAO2ALYP2OVAZFOE3PZHWZWY2HH64MFCO47TINKWA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309824350 (203h20m26.080465675s) [noise=true , address=104.36.82.8:4972 , piece_id=DA5ZEQHEMMJRTFRB4HMJL2OGGSA45P6UPNDJDKAU47S3A6OYQNMA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.310037578 (203h20m26.080678913s) [noise=true , address=104.36.82.4:4804 , piece_id=HV3FD4MSGB3ARPAOJYNYLLXHUS7JAF6KAQWLMIYCXC4UGW5BETTQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.311261871 (203h20m26.081903206s) [noise=true , address=77.90.31.12:4590 , piece_id=5XDULYYTTMHQXP4QSDT4M2W6NS7CM4ZYWH3OU32FMBAGXNQIVU6Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:53.080800025 (203h20m27.85144135s) [upload=162628]: Next returned nil segment Jul 1 20:30:53.080836496 (203h20m27.851477821s) [upload=162628]: Waiting for error group managing segments... Jul 1 20:30:53.362137719 (203h20m28.132806747s) [noise=true , address=45.91.116.4:4937 , piece_id=CMO4GLDKCZ7KQWHM5S4XX5QB2L3KAHUKRUYXWNE5OPEJ464X2NBA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.365537323 (203h20m28.136178648s) [noise=true , address=45.91.116.10:4082 , piece_id=U3OGQJQN3KWNAO2ALYP2OVAZFOE3PZHWZWY2HH64MFCO47TINKWA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.366047188 (203h20m28.136688523s) [noise=true , address=45.91.116.9:5262 , piece_id=N2SGOATWGVAKHJUCGXIUCMO5GTHQS4FHSFXZJBSGGT4HIQK4MFSQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.366428668 (203h20m28.137070003s) [noise=true , address=45.91.116.5:5299 , piece_id=V6X7BVMX3IP23A65VFY7PIHGCJPGKLTZAYH4QVPPUFB34M3JUQYQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.373588435 (203h20m28.14422977s) [noise=true , address=45.91.116.13:4155 , piece_id=WN7BEIS5XO4K3BGAHLFDGBXUK52OHWYBPFFH6GCNADP6HWYBM73Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.395606298 (203h20m28.166247633s) [noise=true , address=77.90.31.12:4589 , piece_id=OAQAOA63GE45RWO52GQFOO6FGD4M2AW6ALCPMKCB7T25UDBFOBIA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.399444591 (203h20m28.170085926s) [noise=true , address=77.90.31.17:4770 , piece_id=OFDOKUGNGNDMRFTF4HF3BHPZO3HSDDJZL7NC6G7PQSPKZRZVWGJA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.409917327 (203h20m28.180558662s) [noise=true , address=77.90.31.12:4590 , piece_id=5XDULYYTTMHQXP4QSDT4M2W6NS7CM4ZYWH3OU32FMBAGXNQIVU6Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.459664931 (203h20m28.230306266s) [noise=true , address=104.36.82.14:4221 , piece_id=JK73AQZVYWALWCBO7DORATYWJOIKWT2WBBHSZ26Q3TJLBYDTYO2Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.461422203 (203h20m28.232063538s) [noise=true , address=45.91.116.13:4154 , piece_id=CWIE4GHPWQCF7YHM2QG62RJMBQBNKNXRK2JI4G6OBTPKN4VV73BA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.506236628 (203h20m28.276877963s) [noise=true , address=45.91.116.6:4975 , piece_id=YUJIO2VMD53TDD66ANUI55LFMDTPHRMKV4Y7L7X66KA2BMKTIUQA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.855073943 (203h20m29.625715268s) [noise=true , address=104.36.82.2:4731 , piece_id=JXQL75TODICFVUAIEE4TRGI2DUYVNC3ZR3RNEARTKQS56IIKZATQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.986719438 (203h20m29.757360773s) [noise=true , address=104.36.82.8:4972 , piece_id=DA5ZEQHEMMJRTFRB4HMJL2OGGSA45P6UPNDJDKAU47S3A6OYQNMA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.986744686 (203h20m29.757386021s) [seg_pos={0 0}, upload=162628]: Segment reached optimal threshold of 13 pieces. Jul 1 20:30:54.986928978 (203h20m29.757570303s) [noise=true , address=45.91.116.8:4048 , piece_id=2G2OOQLJPTUDY5PZSHQ2MNAV7774DPK55SELX5STL5W35MXSFKNQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection Jul 1 20:30:54.986947524 (203h20m29.757588859s) [noise=true , address=104.36.82.7:4909 , piece_id=JNUIUO5MFV5I3L4CTK6IY2CMQDKTVCGPAR6VQHIKE2BKPIQQNGHA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Jul 1 20:30:54.987028379 (203h20m29.757669714s) [noise=true , address=104.36.82.4:4804 , piece_id=HV3FD4MSGB3ARPAOJYNYLLXHUS7JAF6KAQWLMIYCXC4UGW5BETTQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: read tcp 188.166.225.170:50928->104.36.82.4:4804: use of closed network connection; context canceled Jul 1 20:30:54.987039529 (203h20m29.757680864s) [seg_pos={0 0}, upload=162628]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 20:30:54.987097500 (203h20m29.757738835s) [upload=162628]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 20:30:54.987100907 (203h20m29.757742232s) [upload=162628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 20:30:55.290907064 (203h20m30.061548409s) [upload=162628]: Done uploading segments... Jul 1 20:30:55.290920750 (203h20m30.061562085s) [upload=162628]: Upload finished. err: <nil> Duration: 5.770943989s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.065143867s FAIL (2025-07-01 20:31:26 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== p 188.166.225.170:38826->45.91.116.14:4190: use of closed network connection Jul 1 20:30:07.397683164 (203h19m42.168324499s) [seg_pos={0 0}, upload=162621]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 20:30:07.397793445 (203h19m42.16843478s) [upload=162621]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 20:30:07.397832890 (203h19m42.168474225s) [upload=162621]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 20:30:07.690755035 (203h19m42.46139637s) [upload=162621]: Done uploading segments... Jul 1 20:30:07.690777598 (203h19m42.461418933s) [upload=162621]: Upload finished. err: <nil> Jul 1 20:30:51.075923771 (203h20m25.846565096s) [upload=162628]: Starting upload Jul 1 20:30:51.075979769 (203h20m25.846621104s) [upload=162628]: Done starting upload Jul 1 20:30:51.076046185 (203h20m25.84668752s) [upload=162628]: Uploading segments... Jul 1 20:30:51.076156256 (203h20m25.846797591s) [upload=162628]: Got next segment. Inline: false Jul 1 20:30:51.076184720 (203h20m25.846826055s) [upload=162628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 20:30:51.076189279 (203h20m25.846830614s) [upload=162628]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 20:30:51.307264904 (203h20m26.077906239s) [seg_pos={0 0}, upload=162628]: Begin upload segment... Jul 1 20:30:51.307309248 (203h20m26.077950583s) [seg_pos={0 0}, upload=162628]: Done begin upload segment. Jul 1 20:30:51.307323446 (203h20m26.077964771s) [noise=true , address=45.91.116.8:4048 , piece_id=2G2OOQLJPTUDY5PZSHQ2MNAV7774DPK55SELX5STL5W35MXSFKNQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307341239 (203h20m26.077982574s) [noise=true , address=77.90.31.12:4589 , piece_id=OAQAOA63GE45RWO52GQFOO6FGD4M2AW6ALCPMKCB7T25UDBFOBIA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307377640 (203h20m26.078018965s) [noise=true , address=45.91.116.13:4154 , piece_id=CWIE4GHPWQCF7YHM2QG62RJMBQBNKNXRK2JI4G6OBTPKN4VV73BA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307671732 (203h20m26.078313057s) [noise=true , address=104.36.82.14:4221 , piece_id=JK73AQZVYWALWCBO7DORATYWJOIKWT2WBBHSZ26Q3TJLBYDTYO2Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307680439 (203h20m26.078321774s) [noise=true , address=104.36.82.7:4909 , piece_id=JNUIUO5MFV5I3L4CTK6IY2CMQDKTVCGPAR6VQHIKE2BKPIQQNGHA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307944213 (203h20m26.078585548s) [noise=true , address=45.91.116.5:5299 , piece_id=V6X7BVMX3IP23A65VFY7PIHGCJPGKLTZAYH4QVPPUFB34M3JUQYQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308197999 (203h20m26.078839334s) [noise=true , address=104.36.82.2:4731 , piece_id=JXQL75TODICFVUAIEE4TRGI2DUYVNC3ZR3RNEARTKQS56IIKZATQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308448899 (203h20m26.079090234s) [noise=true , address=45.91.116.4:4937 , piece_id=CMO4GLDKCZ7KQWHM5S4XX5QB2L3KAHUKRUYXWNE5OPEJ464X2NBA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308746398 (203h20m26.079387733s) [noise=true , address=77.90.31.17:4770 , piece_id=OFDOKUGNGNDMRFTF4HF3BHPZO3HSDDJZL7NC6G7PQSPKZRZVWGJA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308975577 (203h20m26.079616912s) [noise=true , address=45.91.116.13:4155 , piece_id=WN7BEIS5XO4K3BGAHLFDGBXUK52OHWYBPFFH6GCNADP6HWYBM73Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309167925 (203h20m26.07980926s) [noise=true , address=45.91.116.9:5262 , piece_id=N2SGOATWGVAKHJUCGXIUCMO5GTHQS4FHSFXZJBSGGT4HIQK4MFSQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309359802 (203h20m26.080001137s) [noise=true , address=45.91.116.6:4975 , piece_id=YUJIO2VMD53TDD66ANUI55LFMDTPHRMKV4Y7L7X66KA2BMKTIUQA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309548883 (203h20m26.080190218s) [noise=true , address=45.91.116.10:4082 , piece_id=U3OGQJQN3KWNAO2ALYP2OVAZFOE3PZHWZWY2HH64MFCO47TINKWA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309824350 (203h20m26.080465675s) [noise=true , address=104.36.82.8:4972 , piece_id=DA5ZEQHEMMJRTFRB4HMJL2OGGSA45P6UPNDJDKAU47S3A6OYQNMA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.310037578 (203h20m26.080678913s) [noise=true , address=104.36.82.4:4804 , piece_id=HV3FD4MSGB3ARPAOJYNYLLXHUS7JAF6KAQWLMIYCXC4UGW5BETTQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.311261871 (203h20m26.081903206s) [noise=true , address=77.90.31.12:4590 , piece_id=5XDULYYTTMHQXP4QSDT4M2W6NS7CM4ZYWH3OU32FMBAGXNQIVU6Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:53.080800025 (203h20m27.85144135s) [upload=162628]: Next returned nil segment Jul 1 20:30:53.080836496 (203h20m27.851477821s) [upload=162628]: Waiting for error group managing segments... Jul 1 20:30:53.362137719 (203h20m28.132806747s) [noise=true , address=45.91.116.4:4937 , piece_id=CMO4GLDKCZ7KQWHM5S4XX5QB2L3KAHUKRUYXWNE5OPEJ464X2NBA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.365537323 (203h20m28.136178648s) [noise=true , address=45.91.116.10:4082 , piece_id=U3OGQJQN3KWNAO2ALYP2OVAZFOE3PZHWZWY2HH64MFCO47TINKWA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.366047188 (203h20m28.136688523s) [noise=true , address=45.91.116.9:5262 , piece_id=N2SGOATWGVAKHJUCGXIUCMO5GTHQS4FHSFXZJBSGGT4HIQK4MFSQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.366428668 (203h20m28.137070003s) [noise=true , address=45.91.116.5:5299 , piece_id=V6X7BVMX3IP23A65VFY7PIHGCJPGKLTZAYH4QVPPUFB34M3JUQYQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.373588435 (203h20m28.14422977s) [noise=true , address=45.91.116.13:4155 , piece_id=WN7BEIS5XO4K3BGAHLFDGBXUK52OHWYBPFFH6GCNADP6HWYBM73Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.395606298 (203h20m28.166247633s) [noise=true , address=77.90.31.12:4589 , piece_id=OAQAOA63GE45RWO52GQFOO6FGD4M2AW6ALCPMKCB7T25UDBFOBIA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.399444591 (203h20m28.170085926s) [noise=true , address=77.90.31.17:4770 , piece_id=OFDOKUGNGNDMRFTF4HF3BHPZO3HSDDJZL7NC6G7PQSPKZRZVWGJA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.409917327 (203h20m28.180558662s) [noise=true , address=77.90.31.12:4590 , piece_id=5XDULYYTTMHQXP4QSDT4M2W6NS7CM4ZYWH3OU32FMBAGXNQIVU6Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.459664931 (203h20m28.230306266s) [noise=true , address=104.36.82.14:4221 , piece_id=JK73AQZVYWALWCBO7DORATYWJOIKWT2WBBHSZ26Q3TJLBYDTYO2Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.461422203 (203h20m28.232063538s) [noise=true , address=45.91.116.13:4154 , piece_id=CWIE4GHPWQCF7YHM2QG62RJMBQBNKNXRK2JI4G6OBTPKN4VV73BA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.506236628 (203h20m28.276877963s) [noise=true , address=45.91.116.6:4975 , piece_id=YUJIO2VMD53TDD66ANUI55LFMDTPHRMKV4Y7L7X66KA2BMKTIUQA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.855073943 (203h20m29.625715268s) [noise=true , address=104.36.82.2:4731 , piece_id=JXQL75TODICFVUAIEE4TRGI2DUYVNC3ZR3RNEARTKQS56IIKZATQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.986719438 (203h20m29.757360773s) [noise=true , address=104.36.82.8:4972 , piece_id=DA5ZEQHEMMJRTFRB4HMJL2OGGSA45P6UPNDJDKAU47S3A6OYQNMA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.986744686 (203h20m29.757386021s) [seg_pos={0 0}, upload=162628]: Segment reached optimal threshold of 13 pieces. Jul 1 20:30:54.986928978 (203h20m29.757570303s) [noise=true , address=45.91.116.8:4048 , piece_id=2G2OOQLJPTUDY5PZSHQ2MNAV7774DPK55SELX5STL5W35MXSFKNQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection Jul 1 20:30:54.986947524 (203h20m29.757588859s) [noise=true , address=104.36.82.7:4909 , piece_id=JNUIUO5MFV5I3L4CTK6IY2CMQDKTVCGPAR6VQHIKE2BKPIQQNGHA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Jul 1 20:30:54.987028379 (203h20m29.757669714s) [noise=true , address=104.36.82.4:4804 , piece_id=HV3FD4MSGB3ARPAOJYNYLLXHUS7JAF6KAQWLMIYCXC4UGW5BETTQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: read tcp 188.166.225.170:50928->104.36.82.4:4804: use of closed network connection; context canceled Jul 1 20:30:54.987039529 (203h20m29.757680864s) [seg_pos={0 0}, upload=162628]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 20:30:54.987097500 (203h20m29.757738835s) [upload=162628]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 20:30:54.987100907 (203h20m29.757742232s) [upload=162628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 20:30:55.290907064 (203h20m30.061548409s) [upload=162628]: Done uploading segments... Jul 1 20:30:55.290920750 (203h20m30.061562085s) [upload=162628]: Upload finished. err: <nil> Duration: 6.065168985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001938791s FAIL (2025-07-01 20:31:20 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== p 188.166.225.170:38826->45.91.116.14:4190: use of closed network connection Jul 1 20:30:07.397683164 (203h19m42.168324499s) [seg_pos={0 0}, upload=162621]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 20:30:07.397793445 (203h19m42.16843478s) [upload=162621]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 20:30:07.397832890 (203h19m42.168474225s) [upload=162621]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 20:30:07.690755035 (203h19m42.46139637s) [upload=162621]: Done uploading segments... Jul 1 20:30:07.690777598 (203h19m42.461418933s) [upload=162621]: Upload finished. err: <nil> Jul 1 20:30:51.075923771 (203h20m25.846565096s) [upload=162628]: Starting upload Jul 1 20:30:51.075979769 (203h20m25.846621104s) [upload=162628]: Done starting upload Jul 1 20:30:51.076046185 (203h20m25.84668752s) [upload=162628]: Uploading segments... Jul 1 20:30:51.076156256 (203h20m25.846797591s) [upload=162628]: Got next segment. Inline: false Jul 1 20:30:51.076184720 (203h20m25.846826055s) [upload=162628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 20:30:51.076189279 (203h20m25.846830614s) [upload=162628]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 20:30:51.307264904 (203h20m26.077906239s) [seg_pos={0 0}, upload=162628]: Begin upload segment... Jul 1 20:30:51.307309248 (203h20m26.077950583s) [seg_pos={0 0}, upload=162628]: Done begin upload segment. Jul 1 20:30:51.307323446 (203h20m26.077964771s) [noise=true , address=45.91.116.8:4048 , piece_id=2G2OOQLJPTUDY5PZSHQ2MNAV7774DPK55SELX5STL5W35MXSFKNQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307341239 (203h20m26.077982574s) [noise=true , address=77.90.31.12:4589 , piece_id=OAQAOA63GE45RWO52GQFOO6FGD4M2AW6ALCPMKCB7T25UDBFOBIA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307377640 (203h20m26.078018965s) [noise=true , address=45.91.116.13:4154 , piece_id=CWIE4GHPWQCF7YHM2QG62RJMBQBNKNXRK2JI4G6OBTPKN4VV73BA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307671732 (203h20m26.078313057s) [noise=true , address=104.36.82.14:4221 , piece_id=JK73AQZVYWALWCBO7DORATYWJOIKWT2WBBHSZ26Q3TJLBYDTYO2Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307680439 (203h20m26.078321774s) [noise=true , address=104.36.82.7:4909 , piece_id=JNUIUO5MFV5I3L4CTK6IY2CMQDKTVCGPAR6VQHIKE2BKPIQQNGHA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.307944213 (203h20m26.078585548s) [noise=true , address=45.91.116.5:5299 , piece_id=V6X7BVMX3IP23A65VFY7PIHGCJPGKLTZAYH4QVPPUFB34M3JUQYQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308197999 (203h20m26.078839334s) [noise=true , address=104.36.82.2:4731 , piece_id=JXQL75TODICFVUAIEE4TRGI2DUYVNC3ZR3RNEARTKQS56IIKZATQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308448899 (203h20m26.079090234s) [noise=true , address=45.91.116.4:4937 , piece_id=CMO4GLDKCZ7KQWHM5S4XX5QB2L3KAHUKRUYXWNE5OPEJ464X2NBA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308746398 (203h20m26.079387733s) [noise=true , address=77.90.31.17:4770 , piece_id=OFDOKUGNGNDMRFTF4HF3BHPZO3HSDDJZL7NC6G7PQSPKZRZVWGJA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.308975577 (203h20m26.079616912s) [noise=true , address=45.91.116.13:4155 , piece_id=WN7BEIS5XO4K3BGAHLFDGBXUK52OHWYBPFFH6GCNADP6HWYBM73Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309167925 (203h20m26.07980926s) [noise=true , address=45.91.116.9:5262 , piece_id=N2SGOATWGVAKHJUCGXIUCMO5GTHQS4FHSFXZJBSGGT4HIQK4MFSQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309359802 (203h20m26.080001137s) [noise=true , address=45.91.116.6:4975 , piece_id=YUJIO2VMD53TDD66ANUI55LFMDTPHRMKV4Y7L7X66KA2BMKTIUQA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309548883 (203h20m26.080190218s) [noise=true , address=45.91.116.10:4082 , piece_id=U3OGQJQN3KWNAO2ALYP2OVAZFOE3PZHWZWY2HH64MFCO47TINKWA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.309824350 (203h20m26.080465675s) [noise=true , address=104.36.82.8:4972 , piece_id=DA5ZEQHEMMJRTFRB4HMJL2OGGSA45P6UPNDJDKAU47S3A6OYQNMA, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.310037578 (203h20m26.080678913s) [noise=true , address=104.36.82.4:4804 , piece_id=HV3FD4MSGB3ARPAOJYNYLLXHUS7JAF6KAQWLMIYCXC4UGW5BETTQ, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:51.311261871 (203h20m26.081903206s) [noise=true , address=77.90.31.12:4590 , piece_id=5XDULYYTTMHQXP4QSDT4M2W6NS7CM4ZYWH3OU32FMBAGXNQIVU6Q, seg_pos={0 0}, upload=162628]: Uploading piece... Jul 1 20:30:53.080800025 (203h20m27.85144135s) [upload=162628]: Next returned nil segment Jul 1 20:30:53.080836496 (203h20m27.851477821s) [upload=162628]: Waiting for error group managing segments... Jul 1 20:30:53.362137719 (203h20m28.132806747s) [noise=true , address=45.91.116.4:4937 , piece_id=CMO4GLDKCZ7KQWHM5S4XX5QB2L3KAHUKRUYXWNE5OPEJ464X2NBA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.365537323 (203h20m28.136178648s) [noise=true , address=45.91.116.10:4082 , piece_id=U3OGQJQN3KWNAO2ALYP2OVAZFOE3PZHWZWY2HH64MFCO47TINKWA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.366047188 (203h20m28.136688523s) [noise=true , address=45.91.116.9:5262 , piece_id=N2SGOATWGVAKHJUCGXIUCMO5GTHQS4FHSFXZJBSGGT4HIQK4MFSQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.366428668 (203h20m28.137070003s) [noise=true , address=45.91.116.5:5299 , piece_id=V6X7BVMX3IP23A65VFY7PIHGCJPGKLTZAYH4QVPPUFB34M3JUQYQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.373588435 (203h20m28.14422977s) [noise=true , address=45.91.116.13:4155 , piece_id=WN7BEIS5XO4K3BGAHLFDGBXUK52OHWYBPFFH6GCNADP6HWYBM73Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.395606298 (203h20m28.166247633s) [noise=true , address=77.90.31.12:4589 , piece_id=OAQAOA63GE45RWO52GQFOO6FGD4M2AW6ALCPMKCB7T25UDBFOBIA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.399444591 (203h20m28.170085926s) [noise=true , address=77.90.31.17:4770 , piece_id=OFDOKUGNGNDMRFTF4HF3BHPZO3HSDDJZL7NC6G7PQSPKZRZVWGJA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.409917327 (203h20m28.180558662s) [noise=true , address=77.90.31.12:4590 , piece_id=5XDULYYTTMHQXP4QSDT4M2W6NS7CM4ZYWH3OU32FMBAGXNQIVU6Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.459664931 (203h20m28.230306266s) [noise=true , address=104.36.82.14:4221 , piece_id=JK73AQZVYWALWCBO7DORATYWJOIKWT2WBBHSZ26Q3TJLBYDTYO2Q, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.461422203 (203h20m28.232063538s) [noise=true , address=45.91.116.13:4154 , piece_id=CWIE4GHPWQCF7YHM2QG62RJMBQBNKNXRK2JI4G6OBTPKN4VV73BA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:53.506236628 (203h20m28.276877963s) [noise=true , address=45.91.116.6:4975 , piece_id=YUJIO2VMD53TDD66ANUI55LFMDTPHRMKV4Y7L7X66KA2BMKTIUQA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.855073943 (203h20m29.625715268s) [noise=true , address=104.36.82.2:4731 , piece_id=JXQL75TODICFVUAIEE4TRGI2DUYVNC3ZR3RNEARTKQS56IIKZATQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.986719438 (203h20m29.757360773s) [noise=true , address=104.36.82.8:4972 , piece_id=DA5ZEQHEMMJRTFRB4HMJL2OGGSA45P6UPNDJDKAU47S3A6OYQNMA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: <nil> Jul 1 20:30:54.986744686 (203h20m29.757386021s) [seg_pos={0 0}, upload=162628]: Segment reached optimal threshold of 13 pieces. Jul 1 20:30:54.986928978 (203h20m29.757570303s) [noise=true , address=45.91.116.8:4048 , piece_id=2G2OOQLJPTUDY5PZSHQ2MNAV7774DPK55SELX5STL5W35MXSFKNQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56260->45.91.116.8:4048: use of closed network connection Jul 1 20:30:54.986947524 (203h20m29.757588859s) [noise=true , address=104.36.82.7:4909 , piece_id=JNUIUO5MFV5I3L4CTK6IY2CMQDKTVCGPAR6VQHIKE2BKPIQQNGHA, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Jul 1 20:30:54.987028379 (203h20m29.757669714s) [noise=true , address=104.36.82.4:4804 , piece_id=HV3FD4MSGB3ARPAOJYNYLLXHUS7JAF6KAQWLMIYCXC4UGW5BETTQ, seg_pos={0 0}, upload=162628]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: read tcp 188.166.225.170:50928->104.36.82.4:4804: use of closed network connection; context canceled Jul 1 20:30:54.987039529 (203h20m29.757680864s) [seg_pos={0 0}, upload=162628]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 20:30:54.987097500 (203h20m29.757738835s) [upload=162628]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 20:30:54.987100907 (203h20m29.757742232s) [upload=162628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 20:30:55.290907064 (203h20m30.061548409s) [upload=162628]: Done uploading segments... Jul 1 20:30:55.290920750 (203h20m30.061562085s) [upload=162628]: Upload finished. err: <nil> Duration: 30.001959801s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.757665744s FAIL (2025-07-01 14:11:49 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.757688997s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.688287503s FAIL (2025-07-01 14:10:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.688320226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.678711263s FAIL (2025-07-01 14:10:34 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.67874108s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.7352282s FAIL (2025-07-01 14:09:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.735253287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.738968349s FAIL (2025-07-01 14:09:17 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.738995553s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.870821889s FAIL (2025-07-01 14:08:53 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.87085356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.736524881s FAIL (2025-07-01 14:08:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.73655624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.613575226s FAIL (2025-07-01 14:07:44 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.613604181s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.033240323s FAIL (2025-07-01 14:06:54 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 6.033266051s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.646236714s FAIL (2025-07-01 14:06:10 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.646265349s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.758451699s FAIL (2025-07-01 14:05:04 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.758472569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.776774452s FAIL (2025-07-01 14:04:15 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.776798377s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.680897755s FAIL (2025-07-01 14:03:10 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.680924576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.779104046s FAIL (2025-07-01 14:02:33 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.779133613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.700450836s FAIL (2025-07-01 14:01:39 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.700474901s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.899435468s FAIL (2025-07-01 14:00:51 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.899466908s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.567369657s FAIL (2025-07-01 13:59:55 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.567399434s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.741806052s FAIL (2025-07-01 13:58:52 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.741840738s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.755187903s FAIL (2025-07-01 13:58:12 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.755216798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.644631989s FAIL (2025-07-01 13:57:35 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.644660444s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.711387991s FAIL (2025-07-01 13:56:44 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.711410334s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.890384018s FAIL (2025-07-01 13:55:55 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.890409178s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.71214923s FAIL (2025-07-01 13:54:41 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.712168438s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.829438914s FAIL (2025-07-01 13:53:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.829470525s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.717323561s FAIL (2025-07-01 13:52:29 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.717351544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.764497084s FAIL (2025-07-01 13:51:51 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.764521119s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.682826573s FAIL (2025-07-01 13:51:03 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.682867141s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.674871954s FAIL (2025-07-01 13:50:40 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.674896399s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.325857786s FAIL (2025-07-01 13:49:36 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 6.325885028s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.647223662s FAIL (2025-07-01 13:48:56 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.647247247s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.526488413s FAIL (2025-07-01 13:48:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.526508804s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.595423326s FAIL (2025-07-01 13:48:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.59545136s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.547989153s FAIL (2025-07-01 13:47:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.548018408s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.671929193s FAIL (2025-07-01 13:47:13 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.671957788s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.624336151s FAIL (2025-07-01 13:47:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.624381828s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.636347429s FAIL (2025-07-01 13:46:27 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.636378069s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.543444385s FAIL (2025-07-01 13:46:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.54346746s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.654014518s FAIL (2025-07-01 13:46:01 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.654039976s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.489804441s FAIL (2025-07-01 13:45:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.489825662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.785897337s FAIL (2025-07-01 13:45:08 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.785929869s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.531640512s FAIL (2025-07-01 13:44:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.53166059s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.799050558s FAIL (2025-07-01 13:43:58 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.799082038s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.552825584s FAIL (2025-07-01 13:43:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.552851834s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.777667379s FAIL (2025-07-01 13:42:54 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.777692958s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.123764905s FAIL (2025-07-01 13:42:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.123859105s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.765809475s FAIL (2025-07-01 13:42:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.765883628s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.588061153s FAIL (2025-07-01 13:42:16 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.588085749s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)