Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X.............X.................X.......................] duration:
1.815634519s
FAIL (2025-07-31 08:32:44 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.815660989s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.189485655s
FAIL (2025-07-31 08:32:26 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 5.189526263s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [...................X.....X............XX....................] duration:
5.676855697s
FAIL (2025-07-31 08:32:22 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.676888249s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....X.............X.................X......................] duration:
1.836228785s
FAIL (2025-07-31 08:32:18 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.836252732s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
4.874410381s
FAIL (2025-07-31 08:32:10 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 4.874444065s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [....................X.....X............XX...................] duration:
5.639585522s
FAIL (2025-07-31 08:32:00 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.639614487s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X.............X.................X.....................] duration:
2.087320483s
FAIL (2025-07-31 08:31:46 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.087345261s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....................X.....X............XX..................] duration:
5.644802793s
FAIL (2025-07-31 08:31:22 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.644855764s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.116618875s
FAIL (2025-07-31 08:31:11 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 6.116647822s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [......................X.....X............XX.................] duration:
5.921815029s
FAIL (2025-07-31 08:30:52 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.921839696s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X.............X.................X....................] duration:
1.832707968s
FAIL (2025-07-31 08:30:42 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.832735962s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.......................X.....X............XX................] duration:
5.748283502s
FAIL (2025-07-31 08:30:24 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.748310744s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X.............X.................X...................] duration:
1.898153902s
FAIL (2025-07-31 08:30:13 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.898175883s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
4.976629702s
FAIL (2025-07-31 08:30:05 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 4.976656274s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X.............X.................X..................] duration:
1.852806153s
FAIL (2025-07-31 08:29:29 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.852839837s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
6.062762586s
FAIL (2025-07-31 08:29:27 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 6.062793255s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................X.....X............XX...............] duration:
5.854703191s
FAIL (2025-07-31 08:29:23 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.854731175s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..........X.............X.................X.................] duration:
1.868206257s
FAIL (2025-07-31 08:29:09 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.868236796s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.........................X.....X............XX..............] duration:
5.788276478s
FAIL (2025-07-31 08:28:35 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.788300001s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
6.429763285s
FAIL (2025-07-31 08:28:30 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 6.429797931s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........X.............X.................X................] duration:
1.846751499s
FAIL (2025-07-31 08:28:06 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.846784832s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.011753666s
FAIL (2025-07-31 08:27:49 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 5.011778253s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [..........................X.....X............XX.............] duration:
5.640355435s
FAIL (2025-07-31 08:27:28 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.640384139s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.204725564s
FAIL (2025-07-31 08:27:08 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 5.204751463s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X.............X.................X...............] duration:
1.831167388s
FAIL (2025-07-31 08:27:07 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.8312044s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [...........................X.....X............XX............] duration:
5.725368947s
FAIL (2025-07-31 08:26:50 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.725395868s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............X.............X.................X..............] duration:
1.847575185s
FAIL (2025-07-31 08:26:47 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.847601326s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
4.893378983s
FAIL (2025-07-31 08:26:39 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 4.89340284s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..............X.............X.................X.............] duration:
2.090674066s
FAIL (2025-07-31 08:26:26 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.090698463s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.639772839s
FAIL (2025-07-31 08:26:15 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 5.639802466s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [............................X.....X............XX...........] duration:
5.80680159s
FAIL (2025-07-31 08:25:47 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.806826799s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...............X.............X.................X............] duration:
1.840494025s
FAIL (2025-07-31 08:25:41 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.840520425s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.............................X.....X............XX..........] duration:
5.7706412s
FAIL (2025-07-31 08:25:21 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.770671999s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [................X.............X.................X...........] duration:
1.882767649s
FAIL (2025-07-31 08:25:19 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.882792214s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.120665137s
FAIL (2025-07-31 08:25:10 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 5.120693541s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
4.927653131s
FAIL (2025-07-31 08:24:40 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 4.927673619s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [X.............................X.....X............XX.........] duration:
5.915660566s
FAIL (2025-07-31 08:24:21 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.915721303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.................X.............X.................X..........] duration:
1.875484567s
FAIL (2025-07-31 08:24:18 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.87551806s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.X.............................X.....X............XX........] duration:
5.817683677s
FAIL (2025-07-31 08:23:49 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.81771118s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.661459987s
FAIL (2025-07-31 08:23:35 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 5.661488651s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..X.............................X.....X............XX.......] duration:
5.546489397s
FAIL (2025-07-31 08:23:24 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.546519604s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..................X.............X.................X.........] duration:
1.858417068s
FAIL (2025-07-31 08:23:10 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.858438109s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...X.............................X.....X............XX......] duration:
5.654852685s
FAIL (2025-07-31 08:22:46 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.654876449s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
6.077269804s
FAIL (2025-07-31 08:22:35 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 6.077320782s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
4.902220431s
FAIL (2025-07-31 08:22:16 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 4.902248505s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....X.............................X.....X............XX.....] duration:
5.774904585s
FAIL (2025-07-31 08:22:07 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.774928982s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...................X.............X.................X........] duration:
1.834999021s
FAIL (2025-07-31 08:22:07 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.835031844s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.083801112s
FAIL (2025-07-31 08:21:08 UTC)
orbiter warning:
Operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 17)
======LOGS======
:4041 , piece_id=4TOBZ4RCPXIT2NMXYWFCUOMEOQLPMVBRHMXVQ6OJDYCRHGZPDSRQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.360954037 (13h53m18.298612537s) [noise=true , address=89.149.201.147:4017 , piece_id=SLRL3M6ESSUJH4QJF5KHANYO23UIKQCJHWSJ3ZGGM7K65CWB3CDQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.363716784 (13h53m18.301375283s) [noise=true , address=89.149.201.147:4021 , piece_id=RJNTJZZUC42EQTLG5BJO5SWCCQ7Z53YSUDCG5RHN3ADWQ3PNOREA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.364452280 (13h53m18.302110779s) [noise=true , address=89.149.201.147:4022 , piece_id=QV6PLLEYRPVAVXCKCTJJV4EFFK5OFVGFAO5UFBCSOWZ4RU34QTBQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.366746490 (13h53m18.30440499s) [noise=true , address=81.171.14.70:4047 , piece_id=ID5RSNMKWB5MGKLAQVZ3FLX7I4STJWV2DQ3TYKL64TLMFC5DIZPQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.367408457 (13h53m18.305066956s) [noise=true , address=81.171.14.70:4045 , piece_id=TEYKJIMPN33ITZJBBYNWVK3LX3IH6LW5SSTABUS4HL2RAVGS2SQQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.372749175 (13h53m18.310407674s) [noise=true , address=89.149.201.147:4013 , piece_id=SJR3FB7M6GWJOM3HQ5TXYGGPZ7TVWZG4U2WBEISEKMXRFBTPYTBA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.374657567 (13h53m18.312316076s) [noise=true , address=89.149.201.147:4015 , piece_id=NLA7BDDRIG6BSPR73OR2546ANBFNBKK3KRMQED5YQYQ44PITY2HA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.377442225 (13h53m18.315100724s) [noise=true , address=81.171.14.70:4040 , piece_id=4QVPLHYMIJ4TQMYFXCQBMIVEJKSKAZX3US2RSUTDN5XHYASPEMWQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379210779 (13h53m18.316869278s) [noise=true , address=81.171.14.70:4034 , piece_id=GB7U3E4IDKVU4EK7KUROHY6WYQYP2DBCUOM6WSK43BGDEO6XN6LQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.379742186 (13h53m18.317400685s) [noise=true , address=89.149.201.147:4012 , piece_id=QYIRQVPDR6XMDWW6CUTD3ZTPLVVXF5CHRT75GOFFUWPKW5DEO2IA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.380383443 (13h53m18.318041942s) [noise=true , address=81.171.14.70:4033 , piece_id=KR6SVORADHZ2MYTNGPWNU3EVC7APZCH6B2HAE2OMGW3LGPCQDQFA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382104987 (13h53m18.319763486s) [noise=true , address=89.149.201.147:4024 , piece_id=5GCOSLG6L6BCW7ZUEDFOMZ6ZYO7XQGGUN6J72VU4EFWVUDRLXU3Q, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.382973318 (13h53m18.320631817s) [noise=true , address=81.171.14.70:4029 , piece_id=P66RCMJTQ4INXPSSRIMSIXO4AANJCHZ7HHUTVRIBVBEFBMFMOGYA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.389233987 (13h53m18.326892486s) [noise=true , address=81.171.14.70:4026 , piece_id=C3KI7VEBII5OUW3SVZ25XSIZOCC7N3EYHJLGBTVZKC2RWFOTI5VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392230651 (13h53m18.32988915s) [noise=true , address=81.171.14.70:4043 , piece_id=RAM2YQAKAWEGTUC35KWTA7SRZROBCXS2QHMXDNUCMTQFTRCF7XUA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:50:28.392579929 (13h53m18.330238428s) [noise=true , address=81.171.14.70:4037 , piece_id=7BD4S6F6ATPZRNRNWRAIE7T2R436JQXIWB4KJLM3ZF5F7PLB4GJA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: <nil>
Jul 31 07:51:26.338459026 (13h54m16.276117976s) [noise=true , address=185.167.204.206:4057 , piece_id=RSEBXXPDST27TOP2JPVEH5SSUR7LTYBIMBXJOEFBW2GXPS5AR7YQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338552625 (13h54m16.276211124s) [noise=true , address=185.167.204.206:4067 , piece_id=URM5MRXOTX67D6RUGQQG7RV6ZBH7LEV5HVJNLKE6LHZNUUHT2JOA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, address:185.167.204.206:4067): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338587011 (13h54m16.27624551s) [noise=true , address=185.167.204.206:4071 , piece_id=RJPTJOIYZKUQLKNF3XPE6SXSIPXNRGA2MTN7OYIRFLAA72PFMZ6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34150->185.167.204.206:4071: use of closed network connection
Jul 31 07:51:26.338625304 (13h54m16.276283804s) [noise=true , address=185.167.204.205:4001 , piece_id=OXMN3HLITSTRAC2VZ7EWS237HVTWBCEXGZH7A6UUK66SYZANCKMQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53152->185.167.204.205:4001: use of closed network connection
Jul 31 07:51:26.338629643 (13h54m16.276288142s) [noise=true , address=185.167.204.205:4027 , piece_id=X53P4S5CPPYRPFYMNVYLXBFUTBCXDSUOGUYUHIYPCHOWZUUB2R6A, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338662054 (13h54m16.276320554s) [noise=true , address=185.167.204.206:4059 , piece_id=4HVCGCMYD3YEJSIXVAKZRLVHETHYUFJYDOFS3QIL4FPSU2OG52WA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48448->185.167.204.206:4059: use of closed network connection
Jul 31 07:51:26.338670701 (13h54m16.2763292s) [noise=true , address=185.167.204.206:4043 , piece_id=A5P2HWPQPKVF7FMQTYKEB4PQM4LSP6H2GHEIW2YZTY4I64CSTQGA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: expected piece hash; context deadline exceeded; EOF
Jul 31 07:51:26.338696861 (13h54m16.27635536s) [noise=true , address=185.167.204.206:4051 , piece_id=OKIA4R6LDGESE4BYVGWNH3VRNSUZ47G4GHJ4YKFJHYQWXMWZM4VQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32878->185.167.204.206:4051: use of closed network connection
Jul 31 07:51:26.338731708 (13h54m16.276390207s) [noise=true , address=185.167.204.206:4049 , piece_id=QZNUPVU3FHKDZCHRAMFDRMQAHLME7NDLIKRAX46VK2QBUGUCQHDA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->185.167.204.206:4049: use of closed network connection
Jul 31 07:51:26.338735836 (13h54m16.276394335s) [noise=true , address=185.167.204.205:4009 , piece_id=TJD6X3WZXEKCCWDRNPNQQBYHVAQ2LYHVNWU3LUMHJA346DPIOZHA, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54950->185.167.204.205:4009: use of closed network connection
Jul 31 07:51:26.338773397 (13h54m16.276431897s) [noise=true , address=185.167.204.206:4041 , piece_id=NN467GKNZOQESMERSKWAUNR74IX5B7BPJU7CPR7HYGTSPWEXPYLQ, seg_pos={0 0}, upload=13308]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46116->185.167.204.206:4041: use of closed network connection
Jul 31 07:51:26.338791052 (13h54m16.276449551s) [seg_pos={0 0}, upload=13308]: Done waiting for segment. successful: 17 optimal: 26 errs: <nil>
Jul 31 07:51:26.515116997 (13h54m16.452775486s) [upload=13308]: Done uploading segments...
Jul 31 07:51:26.515145552 (13h54m16.452804051s) [upload=13308]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 17)
Duration: 6.083832724s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....................X.............X.................X.......] duration:
2.013089807s
FAIL (2025-07-31 08:21:06 UTC)
orbiter warning: Operation: ListObjects unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.013119953s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....X.............................X.....X............XX....] duration:
5.982580089s
FAIL (2025-07-31 08:20:57 UTC)
orbiter warning: Operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 7722db08c632d0b77d6a2cf33c88a45481a859049bde9d0bd48b256d337ace9d != before 97da0a7176e64d019c76c44e503770db5f42523790778397e3ded4780a832b9b
Duration: 5.98260689s
core
uplink
us-select-1
us1-placement-20
|