Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.............X......X..X...X........................X.......] duration:
8.105734293s
FAIL (2025-08-11 04:03:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 8.10575945s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....X.....................X..................X...X..........] duration:
1.835780054s
FAIL (2025-08-11 04:03:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.835804791s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..............X......X..X...X........................X......] duration:
8.19561633s
FAIL (2025-08-11 04:02:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 8.195647991s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....X.....................X..................X...X.........] duration:
1.838341466s
FAIL (2025-08-11 04:02:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.838378025s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...............X......X..X...X........................X.....] duration:
8.079833482s
FAIL (2025-08-11 04:02:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 8.079862629s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X...............X......X..X...X........................X....] duration:
8.239586741s
FAIL (2025-08-11 04:02:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 8.239612711s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [......X.....................X..................X...X........] duration:
1.824887934s
FAIL (2025-08-11 04:01:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.824927189s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X...............X......X..X...X........................X...] duration:
7.771295293s
FAIL (2025-08-11 04:01:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 7.771338134s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X...............X......X..X...X........................X..] duration:
8.911958521s
FAIL (2025-08-11 04:01:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 8.911987518s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
9.102891189s
FAIL (2025-08-11 04:00:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.102909052s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......X.....................X..................X...X.......] duration:
1.854445568s
FAIL (2025-08-11 04:00:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.854479383s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [........X.....................X..................X...X......] duration:
1.856424803s
FAIL (2025-08-11 04:00:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.856455492s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X...............X......X..X...X........................X.] duration:
7.983864398s
FAIL (2025-08-11 03:59:58 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 7.983888264s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.894955187s
FAIL (2025-08-11 03:59:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.894970566s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....X...............X......X..X...X........................X] duration:
7.352566389s
FAIL (2025-08-11 03:59:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2f7fabf87276f927d3a97ee573f4068ce474a4188e6d53a41e9556e729f6b81c != before 55893ef04fabf6f84c91adc979510a70261b75f83eeeca3e1b8029ff281f65dc
======LOGS======
Aug 11 03:59:16.225944509 (58h33m10.65702464s) [upload=61023]: Starting upload
Aug 11 03:59:16.225987852 (58h33m10.657067983s) [upload=61023]: Done starting upload
Aug 11 03:59:16.226084617 (58h33m10.657164748s) [upload=61023]: Uploading segments...
Aug 11 03:59:16.226159009 (58h33m10.65723914s) [upload=61023]: Got next segment. Inline: false
Aug 11 03:59:16.226190059 (58h33m10.65727019s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:59:16.226196040 (58h33m10.657276171s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 03:59:16.679217558 (58h33m11.110297689s) [seg_pos={0 0}, upload=61023]: Begin upload segment...
Aug 11 03:59:16.679299815 (58h33m11.110379946s) [seg_pos={0 0}, upload=61023]: Done begin upload segment.
Aug 11 03:59:16.679366382 (58h33m11.110446553s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679832815 (58h33m11.110912946s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.679927937 (58h33m11.111008068s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680067845 (58h33m11.111147976s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680482418 (58h33m11.111562539s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680708621 (58h33m11.111788752s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.680988576 (58h33m11.112068717s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681212324 (58h33m11.112292455s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681563867 (58h33m11.112643998s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.681682915 (58h33m11.112763046s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682008127 (58h33m11.113088248s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682208892 (58h33m11.113289023s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.682397563 (58h33m11.113477694s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.685996558 (58h33m11.117076689s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.686422143 (58h33m11.117502274s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.687814066 (58h33m11.118894197s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688129890 (58h33m11.119210021s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:16.688406890 (58h33m11.119487101s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Uploading piece...
Aug 11 03:59:18.327217757 (58h33m12.758297898s) [upload=61023]: Next returned nil segment
Aug 11 03:59:18.327269595 (58h33m12.758349726s) [upload=61023]: Waiting for error group managing segments...
Aug 11 03:59:18.903426032 (58h33m13.334506163s) [noise=true , address=104.36.82.39:4743 , piece_id=OUW7RRH3MSDCA6GS2Z5F6BBLVJPNAKOIXNIT4J3RNT6B2ZES5XHA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.005340670 (58h33m13.436420811s) [noise=true , address=104.36.82.9:4015 , piece_id=NNEV7FA2AJHP4GOLMLEYCTBFJQZ5COKQCA4FTEITNA7NU6EMRTMA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.087428471 (58h33m13.518508592s) [noise=true , address=104.36.82.14:4243 , piece_id=XLOYXDZZZTKYQCBMS7P26FRPKHSWSOLQIBBVM4UL6NOKBGLVXPAQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.301702909 (58h33m13.73278304s) [noise=true , address=104.36.82.58:4286 , piece_id=ZHHNDHLIFZLCRRLXQXHBXIKP26A3CLGCNZBYHFBDCBI5ZXEAYCGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.329314218 (58h33m13.760394339s) [noise=true , address=104.36.82.37:4664 , piece_id=YBFHIZ4PEM7ZZHP2NI6KEWS7WGN63EP633D5XV63J6N2BZ4P3YDQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.345588877 (58h33m13.776669008s) [noise=true , address=104.36.82.60:4366 , piece_id=5YEWG3BVXTJIEGGN5GN3D5J4OASLSGHELCLL32YX2WN544VYQRNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.373969789 (58h33m13.80504991s) [noise=true , address=104.36.82.44:4901 , piece_id=RV3K2QIV5PZGHQRNZZDO7BDQIF3GUHZAOG6LXQBUVIY7EA75HUGA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.447358676 (58h33m13.878438807s) [noise=true , address=104.36.82.16:4328 , piece_id=FPACWYXYNQLQGNUJIHRC3APZCTKBNYICZLUNSMAALZABAA7Z6PVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.502399528 (58h33m13.933479649s) [noise=true , address=104.36.82.5:4843 , piece_id=IPQAJUZGCKFLJTMAGEHCRIMIQILKXYFDIHMEZSQCKFSCJCRBVPRQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.715956382 (58h33m14.147036513s) [noise=true , address=104.36.82.52:4103 , piece_id=CE5ZHI2QYNLBPGAEJODBTJ2P7SY3YDKLGFJAYHPWOGOSTHZQSBLA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.743410991 (58h33m14.174491163s) [noise=true , address=104.36.82.17:4363 , piece_id=6PAE6L2QMZPHUK2KRXZJRKLEKQMW5MEX74TVO55RXTTS3B3YRMQQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.765163221 (58h33m14.196243342s) [noise=true , address=104.36.82.34:4569 , piece_id=5MP2EAZINL4NMY6AZIKCLCJHQYTYVBV2HR66HMPMCGPHOERAZC5Q, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:19.859252466 (58h33m14.290332597s) [noise=true , address=104.36.82.36:4644 , piece_id=NVT4DJJDG43HD23HBIOVFR6BSXEZL3ZRLOPHYNDUMR3WWCMKEF7A, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172139773 (58h33m14.603219904s) [noise=true , address=45.91.116.14:4217 , piece_id=H5GDARCCYR7VYAIPGNAF7MECOQT7UQWXQQ33ZY6KNGEALR4BGDNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: <nil>
Aug 11 03:59:20.172173437 (58h33m14.603253568s) [seg_pos={0 0}, upload=61023]: Segment reached optimal threshold of 14 pieces.
Aug 11 03:59:20.172297275 (58h33m14.603377406s) [noise=true , address=104.36.82.19:4434 , piece_id=7FSJPLS5VKJGV66DZSPM65BQFEGA7ANOKMSFCP3UBEEORH5CE2GA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hC4vAqz6Twt4iWiySS8pxktAKkiDfJAoRTSjcUk8oYgeC8GDt): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172355606 (58h33m14.603435737s) [noise=true , address=104.36.82.61:4378 , piece_id=7YDOWP2RT7HHRVJVR6WLSCVUUKWDZTK4T67WPSU6V66IKE5X2GNQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172413116 (58h33m14.603493237s) [noise=true , address=104.36.82.38:4709 , piece_id=OTQWHTWM6TN36PCVLWBD45YSW7NCRIAWAYEJ32QQR6JLHLE75ZTA, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ad77dgyCKuYL8o9Ac1WrMfk7pG1mgkakcVXwZa2QHD6KG5sdH): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172436872 (58h33m14.603517003s) [noise=true , address=104.36.82.59:4318 , piece_id=3OF62EQJERTJLC3BELUURTY77SAX43XES7QC2E7HJR7PHRFXVAVQ, seg_pos={0 0}, upload=61023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF
Aug 11 03:59:20.172486356 (58h33m14.603566487s) [seg_pos={0 0}, upload=61023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 11 03:59:20.172575146 (58h33m14.603655277s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 11 03:59:20.172579554 (58h33m14.603659685s) [upload=61023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:59:20.489628835 (58h33m14.920708956s) [upload=61023]: Done uploading segments...
Aug 11 03:59:20.489665004 (58h33m14.920745135s) [upload=61023]: Upload finished. err: <nil>
Duration: 7.352590766s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.........X.....................X..................X...X.....] duration:
2.242802934s
FAIL (2025-08-11 03:59:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 2.242879861s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.78719902s
FAIL (2025-08-11 03:59:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.787217226s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....X...............X......X..X...X........................] duration:
7.446042468s
FAIL (2025-08-11 03:58:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.446071554s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..........X.....................X..................X...X....] duration:
1.822990506s
FAIL (2025-08-11 03:58:26 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.823016144s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.897461501s
FAIL (2025-08-11 03:58:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.897487381s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......X...............X......X..X...X.......................] duration:
8.042327665s
FAIL (2025-08-11 03:57:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.042354788s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...........X.....................X..................X...X...] duration:
1.823523884s
FAIL (2025-08-11 03:57:30 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.823549672s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......X...............X......X..X...X......................] duration:
8.433856606s
FAIL (2025-08-11 03:57:24 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.433892504s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.148332516s
FAIL (2025-08-11 03:56:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.148357545s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [............X.....................X..................X...X..] duration:
1.922050664s
FAIL (2025-08-11 03:56:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.922077314s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........X...............X......X..X...X.....................] duration:
8.647874934s
FAIL (2025-08-11 03:56:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.647902375s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X....X......................................................] duration:
6.490038297s
FAIL (2025-08-11 03:55:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.49005506s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X....X.....................................................] duration:
6.759509996s
FAIL (2025-08-11 03:55:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.759532509s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.............X.....................X..................X...X.] duration:
1.809903291s
FAIL (2025-08-11 03:55:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.809938228s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X........X...............X......X..X...X....................] duration:
8.823193473s
FAIL (2025-08-11 03:55:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.823261643s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X....X....................................................] duration:
8.146823337s
FAIL (2025-08-11 03:54:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.146848895s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X....X...................................................] duration:
6.108544006s
FAIL (2025-08-11 03:54:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.108593259s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [X.............X.....................X..................X...X] duration:
1.75144981s
FAIL (2025-08-11 03:54:15 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 45d86058c9d2c141b8363fd79afd7c4325c931b37dacc929fa69bc21bffbc977 != before bb708323aebff2156ad800c07ed499f6b01bb3d75b1c993011fcedd883c1dc13 ======LOGS====== Aug 11 03:54:14.847337518 (58h28m9.278417659s) [upload=60939]: Starting upload Aug 11 03:54:14.847394036 (58h28m9.278474167s) [upload=60939]: Done starting upload Aug 11 03:54:14.847456666 (58h28m9.278536797s) [upload=60939]: Uploading segments... Aug 11 03:54:14.847502614 (58h28m9.278582745s) [upload=60939]: Got next segment. Inline: true Aug 11 03:54:14.847507092 (58h28m9.278587224s) [upload=60939]: Waiting for error group managing segments... Aug 11 03:54:14.847551338 (58h28m9.278631469s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:54:14.847560425 (58h28m9.278640586s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 03:54:14.847566847 (58h28m9.278646978s) [upload=60939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 03:54:15.280793915 (58h28m9.711874046s) [upload=60939]: Done uploading segments... Aug 11 03:54:15.280809846 (58h28m9.711889977s) [upload=60939]: Upload finished. err: <nil>
Duration: 1.751470981s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X........X...............X......X..X...X...................] duration:
8.029012216s
FAIL (2025-08-11 03:54:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.029039669s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X....X..................................................] duration:
6.399389581s
FAIL (2025-08-11 03:54:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.399413737s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.X.............X.....................X..................X...] duration:
1.880243599s
FAIL (2025-08-11 03:53:49 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 11 03:51:37.980026665 (58h25m32.411106796s) [upload=60896]: Starting upload
Aug 11 03:51:37.980067973 (58h25m32.411148104s) [upload=60896]: Done starting upload
Aug 11 03:51:37.980123910 (58h25m32.411204041s) [upload=60896]: Uploading segments...
Aug 11 03:51:37.980200187 (58h25m32.411280308s) [upload=60896]: Got next segment. Inline: true
Aug 11 03:51:37.980204374 (58h25m32.411284505s) [upload=60896]: Waiting for error group managing segments...
Aug 11 03:51:37.980236947 (58h25m32.411317078s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:51:37.980241235 (58h25m32.411321366s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 11 03:51:37.980243911 (58h25m32.411324042s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:51:38.259759340 (58h25m32.690839481s) [upload=60896]: Done uploading segments...
Aug 11 03:51:38.259783085 (58h25m32.690863216s) [upload=60896]: Upload finished. err: <nil>
Duration: 1.88027036s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X....X.................................................] duration:
5.831063409s
FAIL (2025-08-11 03:53:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.831084851s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..X.............X.....................X..................X..] duration:
1.822259943s
FAIL (2025-08-11 03:53:06 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 11 03:51:37.980026665 (58h25m32.411106796s) [upload=60896]: Starting upload
Aug 11 03:51:37.980067973 (58h25m32.411148104s) [upload=60896]: Done starting upload
Aug 11 03:51:37.980123910 (58h25m32.411204041s) [upload=60896]: Uploading segments...
Aug 11 03:51:37.980200187 (58h25m32.411280308s) [upload=60896]: Got next segment. Inline: true
Aug 11 03:51:37.980204374 (58h25m32.411284505s) [upload=60896]: Waiting for error group managing segments...
Aug 11 03:51:37.980236947 (58h25m32.411317078s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:51:37.980241235 (58h25m32.411321366s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 11 03:51:37.980243911 (58h25m32.411324042s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:51:38.259759340 (58h25m32.690839481s) [upload=60896]: Done uploading segments...
Aug 11 03:51:38.259783085 (58h25m32.690863216s) [upload=60896]: Upload finished. err: <nil>
Duration: 1.822288679s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X........X...............X......X..X...X..................] duration:
7.464230557s
FAIL (2025-08-11 03:52:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.46425838s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...X.............X.....................X..................X.] duration:
1.839963631s
FAIL (2025-08-11 03:52:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 11 03:51:37.980026665 (58h25m32.411106796s) [upload=60896]: Starting upload
Aug 11 03:51:37.980067973 (58h25m32.411148104s) [upload=60896]: Done starting upload
Aug 11 03:51:37.980123910 (58h25m32.411204041s) [upload=60896]: Uploading segments...
Aug 11 03:51:37.980200187 (58h25m32.411280308s) [upload=60896]: Got next segment. Inline: true
Aug 11 03:51:37.980204374 (58h25m32.411284505s) [upload=60896]: Waiting for error group managing segments...
Aug 11 03:51:37.980236947 (58h25m32.411317078s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:51:37.980241235 (58h25m32.411321366s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 11 03:51:37.980243911 (58h25m32.411324042s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:51:38.259759340 (58h25m32.690839481s) [upload=60896]: Done uploading segments...
Aug 11 03:51:38.259783085 (58h25m32.690863216s) [upload=60896]: Upload finished. err: <nil>
Duration: 1.839989629s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X........X...............X......X..X...X.................] duration:
7.52051434s
FAIL (2025-08-11 03:52:26 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.52054019s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X....X................................................] duration:
6.060674247s
FAIL (2025-08-11 03:52:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.060700448s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....X........X...............X......X..X...X................] duration:
8.203319994s
FAIL (2025-08-11 03:52:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.203344852s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X....X...............................................] duration:
8.055004539s
FAIL (2025-08-11 03:51:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.055024536s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....X.............X.....................X..................X] duration:
1.629473841s
FAIL (2025-08-11 03:51:38 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 11 03:51:37.980026665 (58h25m32.411106796s) [upload=60896]: Starting upload
Aug 11 03:51:37.980067973 (58h25m32.411148104s) [upload=60896]: Done starting upload
Aug 11 03:51:37.980123910 (58h25m32.411204041s) [upload=60896]: Uploading segments...
Aug 11 03:51:37.980200187 (58h25m32.411280308s) [upload=60896]: Got next segment. Inline: true
Aug 11 03:51:37.980204374 (58h25m32.411284505s) [upload=60896]: Waiting for error group managing segments...
Aug 11 03:51:37.980236947 (58h25m32.411317078s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:51:37.980241235 (58h25m32.411321366s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 11 03:51:37.980243911 (58h25m32.411324042s) [upload=60896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:51:38.259759340 (58h25m32.690839481s) [upload=60896]: Done uploading segments...
Aug 11 03:51:38.259783085 (58h25m32.690863216s) [upload=60896]: Upload finished. err: <nil>
Duration: 1.629498287s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....X........X...............X......X..X...X...............] duration:
7.892793769s
FAIL (2025-08-11 03:51:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.892820079s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [X....X.............X.....................X..................] duration:
1.822430471s
FAIL (2025-08-11 03:50:53 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 11 03:35:59.697389440 (58h9m54.128469571s) [upload=60624]: Starting upload
Aug 11 03:35:59.697433665 (58h9m54.128513796s) [upload=60624]: Done starting upload
Aug 11 03:35:59.697484944 (58h9m54.128565075s) [upload=60624]: Uploading segments...
Aug 11 03:35:59.697523367 (58h9m54.128603488s) [upload=60624]: Got next segment. Inline: true
Aug 11 03:35:59.697527264 (58h9m54.128607395s) [upload=60624]: Waiting for error group managing segments...
Aug 11 03:35:59.697564615 (58h9m54.128644736s) [upload=60624]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:35:59.697568724 (58h9m54.128648855s) [upload=60624]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 11 03:35:59.697571459 (58h9m54.12865159s) [upload=60624]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:35:59.988045132 (58h9m54.419125263s) [upload=60624]: Done uploading segments...
Aug 11 03:35:59.988063237 (58h9m54.419143358s) [upload=60624]: Upload finished. err: <nil>
Duration: 1.822460387s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X....X..............................................] duration:
6.200957363s
FAIL (2025-08-11 03:50:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.200993643s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [......X........X...............X......X..X...X..............] duration:
8.356241895s
FAIL (2025-08-11 03:50:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 03:39:30.119006289 (58h13m24.55008641s) [upload=60686]: Starting upload Aug 11 03:39:30.119057457 (58h13m24.550137578s) [upload=60686]: Done starting upload Aug 11 03:39:30.119224998 (58h13m24.550305129s) [upload=60686]: Uploading segments... Aug 11 03:39:30.119335248 (58h13m24.55041538s) [upload=60686]: Got next segment. Inline: false Aug 11 03:39:30.119382719 (58h13m24.55046285s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 03:39:30.119387358 (58h13m24.550467489s) [upload=60686]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 03:39:30.358540629 (58h13m24.78962076s) [upload=60686]: Next segment err: metaclient: Too Many Requests Aug 11 03:39:30.358572319 (58h13m24.78965249s) [upload=60686]: Done uploading segments... Aug 11 03:39:30.358598930 (58h13m24.789679081s) [upload=60686]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.356266992s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.X....X.............X.....................X.................] duration:
1.917239186s
FAIL (2025-08-11 03:50:16 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 11 03:35:59.697389440 (58h9m54.128469571s) [upload=60624]: Starting upload
Aug 11 03:35:59.697433665 (58h9m54.128513796s) [upload=60624]: Done starting upload
Aug 11 03:35:59.697484944 (58h9m54.128565075s) [upload=60624]: Uploading segments...
Aug 11 03:35:59.697523367 (58h9m54.128603488s) [upload=60624]: Got next segment. Inline: true
Aug 11 03:35:59.697527264 (58h9m54.128607395s) [upload=60624]: Waiting for error group managing segments...
Aug 11 03:35:59.697564615 (58h9m54.128644736s) [upload=60624]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 03:35:59.697568724 (58h9m54.128648855s) [upload=60624]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 11 03:35:59.697571459 (58h9m54.12865159s) [upload=60624]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 11 03:35:59.988045132 (58h9m54.419125263s) [upload=60624]: Done uploading segments...
Aug 11 03:35:59.988063237 (58h9m54.419143358s) [upload=60624]: Upload finished. err: <nil>
Duration: 1.91726712s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|