Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.669237099s FAIL (2025-07-25 10:20:02 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.6692634s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.146348996s FAIL (2025-07-25 10:19:59 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 2.146369816s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 1.953121909s FAIL (2025-07-25 10:19:57 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.953152037s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X........................X..............................] duration: 1.939026008s FAIL (2025-07-25 10:19:36 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.939056026s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.850942774s FAIL (2025-07-25 10:19:24 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.850965276s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.558860177s FAIL (2025-07-25 10:19:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.558883682s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.979005098s FAIL (2025-07-25 10:19:20 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.979029205s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X....X........................X.............................] duration: 1.894181338s FAIL (2025-07-25 10:19:13 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.894203861s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.906004902s FAIL (2025-07-25 10:19:12 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.906036303s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.X........................................................] duration: 2.134505245s FAIL (2025-07-25 10:19:10 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.13452894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XX...........................................X............] duration: 5.809529161s FAIL (2025-07-25 10:19:03 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/4M Duration: 5.809558689s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.98753379s FAIL (2025-07-25 10:18:45 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.987558679s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.726430756s FAIL (2025-07-25 10:18:28 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.726459111s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.536792457s FAIL (2025-07-25 10:18:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.536808626s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.899991602s FAIL (2025-07-25 10:18:23 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.90001142s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.703879588s FAIL (2025-07-25 10:18:21 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.703907712s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XX...........................................X...........] duration: 5.947427286s FAIL (2025-07-25 10:18:14 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/4M Duration: 5.947450691s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X....X........................X............................] duration: 1.928684471s FAIL (2025-07-25 10:18:13 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.928711423s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.495760892s FAIL (2025-07-25 10:18:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.495777764s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration: 2.021248499s FAIL (2025-07-25 10:17:59 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.021286342s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.866166848s FAIL (2025-07-25 10:17:58 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.866187749s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X....X........................X...........................] duration: 1.881486386s FAIL (2025-07-25 10:17:47 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.881513017s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.056849837s FAIL (2025-07-25 10:17:41 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 6.056868922s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.758095513s FAIL (2025-07-25 10:17:39 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.758124458s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.081404852s FAIL (2025-07-25 10:17:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.081426604s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.979538583s FAIL (2025-07-25 10:17:37 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.97956283s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X.X......................................................] duration: 1.932544818s FAIL (2025-07-25 10:17:31 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.932567191s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XX...........................................X..........] duration: 5.684716593s FAIL (2025-07-25 10:17:22 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/4M Duration: 5.684742983s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.909187162s FAIL (2025-07-25 10:17:12 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.90921762s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.87092353s FAIL (2025-07-25 10:17:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.8709444s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.4334972s FAIL (2025-07-25 10:17:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.433520523s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 5.731190323s FAIL (2025-07-25 10:17:08 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.731218889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X....X........................X..........................] duration: 1.844749176s FAIL (2025-07-25 10:16:55 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.844780727s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration: 2.010639157s FAIL (2025-07-25 10:16:44 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.010669355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.838830928s FAIL (2025-07-25 10:16:37 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.838858359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.945040547s FAIL (2025-07-25 10:16:37 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.945071806s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X....X........................X.........................] duration: 1.831431905s FAIL (2025-07-25 10:16:30 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.831459348s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X....XX...........................................X.........] duration: 5.795067945s FAIL (2025-07-25 10:16:27 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/4M Duration: 5.795090679s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.451736015s FAIL (2025-07-25 10:16:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.451760222s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 5.829822659s FAIL (2025-07-25 10:16:09 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.829848397s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X....X........................X........................] duration: 2.117649715s FAIL (2025-07-25 10:16:04 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 2.11768901s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.725733105s FAIL (2025-07-25 10:16:01 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.725765187s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.184490181s FAIL (2025-07-25 10:15:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.184514167s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.071471954s FAIL (2025-07-25 10:15:54 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 2.071508955s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X....XX...........................................X........] duration: 6.019274167s FAIL (2025-07-25 10:15:51 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/4M Duration: 6.019302181s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......X....X........................X.......................] duration: 1.878397771s FAIL (2025-07-25 10:15:46 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.878428278s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.687167154s FAIL (2025-07-25 10:15:45 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.687196s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.438261586s FAIL (2025-07-25 10:15:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.438304479s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration: 1.916833034s FAIL (2025-07-25 10:15:37 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.916861238s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.88611659s FAIL (2025-07-25 10:15:31 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.886144062s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)