Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.876374277s FAIL (2025-08-07 14:04:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.876400357s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.041852189s FAIL (2025-08-07 14:04:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 2.041869983s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X..................X.........................X......XX..] duration: 6.04079088s FAIL (2025-08-07 14:03:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 379a1e9654a2e11db197cba6085d9ca1a53aa639aef84e2a9b0d218a2128f1ca != before 3d2fb1b55f8ebd735fbff3c20592472b8ca42b215f2e645245a8bfe10574650b ======LOGS====== finished. err: <nil> Aug 7 14:01:55.179966005 (4h21m54.769310993s) [upload=4560]: Starting upload Aug 7 14:01:55.180027874 (4h21m54.769372861s) [upload=4560]: Done starting upload Aug 7 14:01:55.180144266 (4h21m54.769489253s) [upload=4560]: Uploading segments... Aug 7 14:01:55.180380849 (4h21m54.769725836s) [upload=4560]: Got next segment. Inline: false Aug 7 14:01:55.180414263 (4h21m54.76975925s) [upload=4560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 14:01:55.180418821 (4h21m54.769763808s) [upload=4560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 14:01:55.696218378 (4h21m55.285563365s) [seg_pos={0 0}, upload=4560]: Begin upload segment... Aug 7 14:01:55.696261209 (4h21m55.285606196s) [seg_pos={0 0}, upload=4560]: Done begin upload segment. Aug 7 14:01:55.696285175 (4h21m55.285630162s) [noise=true , address=104.36.82.37:4657 , piece_id=THIARAH55CX7JNZVQQPYI53EYMMXYIIKLA6OZALSIIIPMXKUERIQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696421506 (4h21m55.285766503s) [noise=true , address=104.36.82.36:4648 , piece_id=ZNTL7IT3QKPJWLY7CK3XTMZFNIUQFORSKQ6NXYWBGLORRYD3J66A, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696700209 (4h21m55.286045186s) [noise=true , address=104.36.82.59:4304 , piece_id=5HAYGO6W4DJUF4RLOIAFNLBEDLQ3RZTIUNQV7USEIOMXJFKJJZDQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696898969 (4h21m55.286243956s) [noise=true , address=104.36.82.16:4328 , piece_id=OZXU7U6ITSX2TGNZMQEI5XXOP4IEIFVLD4JQJUJYXP26CKMXD7PQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696993150 (4h21m55.286338137s) [noise=true , address=64.49.13.155:4914 , piece_id=4WTEGNBHLDKT7FU4ZHBOXMCGCQEQZOECCDXZ36AIRBFXK7HJMEMA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697235754 (4h21m55.286580741s) [noise=true , address=104.36.82.47:4543 , piece_id=ADMNLQJAAL3LBEJKVMRXPE3YD7LQWDNCZZ7NJMZ52ZRPD6F3XKJQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697291010 (4h21m55.286635997s) [noise=true , address=45.91.116.11:4124 , piece_id=ZF2NWECU4ECYBK5LBM57EWNGVZK4GP5TRPHDJN3NCDM564R3GVLA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697532753 (4h21m55.28687774s) [noise=true , address=45.91.116.7:4015 , piece_id=UWCY5JCZXORDTW2O7452QPNUOW5G5WO3IFAOHQ4Y7VWCGKOTOJDA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697573731 (4h21m55.286918718s) [noise=true , address=104.36.82.34:4565 , piece_id=5554GUPBKMWBZLSYBSCVGT62VPHLAU6KQKJZ4D3RQGUGYBQTKSWQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697921356 (4h21m55.287266343s) [noise=true , address=104.36.82.58:4276 , piece_id=S365XPUFKRWCH2R73XRSITWS67PBAVJ5HWZRQQNXVLWY3DTJQE7Q, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697988324 (4h21m55.287333311s) [noise=true , address=104.36.82.61:4381 , piece_id=FC7OY57PJSFEXISS7ATZLBKF5ROYJ46IMRLFJJN3JTZZAEESYPTA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698150995 (4h21m55.287495982s) [noise=true , address=45.91.116.13:4184 , piece_id=ZLMZYZFI66EKCAAGOQFD5KFNIAH2GGLI2ZQAGS74QSDP726OJ2QQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698310741 (4h21m55.287655728s) [noise=true , address=104.36.82.9:4009 , piece_id=HXBANDR2AIXFHWYVE7Q5HRSZCD6MB63MS5CFIGSSYF27Z4NBDURQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698446340 (4h21m55.287791328s) [noise=true , address=104.36.82.62:4422 , piece_id=CUKWVUNPGDNPJ53IULXXROM7FMYTA5C3GNBIGYMHQHEMTRKJMPYA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698593622 (4h21m55.287938609s) [noise=true , address=104.36.82.23:4451 , piece_id=ZIEI4KSNE3ZUB2EAAEEMP3TEQU6JLU22EVXYD6HALCODCR6M2W7Q, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698671090 (4h21m55.288016077s) [noise=true , address=104.36.82.8:4974 , piece_id=FMUFZCB73EPXFNONS4T5HVYIR5LJONVVG2HDSMBUDZPPGKMGG2IQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698843219 (4h21m55.288188206s) [noise=true , address=104.36.82.15:4281 , piece_id=QXGWPJN2TRKVD3NL3D37FLSUIE5VAFEUIWASXLOMGAZQ6OUFJSDQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.699065424 (4h21m55.288410411s) [noise=true , address=104.36.82.12:4170 , piece_id=JNYFNJG2JMCIWGOWNH2KFLR4TAYUHBAG2PLVOWPFY3NPGS3XK7CA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:56.874315860 (4h21m56.463660847s) [upload=4560]: Next returned nil segment Aug 7 14:01:56.874383459 (4h21m56.463728446s) [upload=4560]: Waiting for error group managing segments... Aug 7 14:01:57.175871398 (4h21m56.765216385s) [noise=true , address=64.49.13.155:4914 , piece_id=4WTEGNBHLDKT7FU4ZHBOXMCGCQEQZOECCDXZ36AIRBFXK7HJMEMA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.621709858 (4h21m57.211054845s) [noise=true , address=104.36.82.23:4451 , piece_id=ZIEI4KSNE3ZUB2EAAEEMP3TEQU6JLU22EVXYD6HALCODCR6M2W7Q, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.634524196 (4h21m57.223869183s) [noise=true , address=104.36.82.36:4648 , piece_id=ZNTL7IT3QKPJWLY7CK3XTMZFNIUQFORSKQ6NXYWBGLORRYD3J66A, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.639033634 (4h21m57.228378621s) [noise=true , address=104.36.82.62:4422 , piece_id=CUKWVUNPGDNPJ53IULXXROM7FMYTA5C3GNBIGYMHQHEMTRKJMPYA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.649834548 (4h21m57.239179545s) [noise=true , address=104.36.82.34:4565 , piece_id=5554GUPBKMWBZLSYBSCVGT62VPHLAU6KQKJZ4D3RQGUGYBQTKSWQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.669716348 (4h21m57.259061335s) [noise=true , address=45.91.116.13:4184 , piece_id=ZLMZYZFI66EKCAAGOQFD5KFNIAH2GGLI2ZQAGS74QSDP726OJ2QQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.686125904 (4h21m57.275470891s) [noise=true , address=45.91.116.11:4124 , piece_id=ZF2NWECU4ECYBK5LBM57EWNGVZK4GP5TRPHDJN3NCDM564R3GVLA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.687869821 (4h21m57.277214808s) [noise=true , address=45.91.116.7:4015 , piece_id=UWCY5JCZXORDTW2O7452QPNUOW5G5WO3IFAOHQ4Y7VWCGKOTOJDA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.697242272 (4h21m57.286587259s) [noise=true , address=104.36.82.12:4170 , piece_id=JNYFNJG2JMCIWGOWNH2KFLR4TAYUHBAG2PLVOWPFY3NPGS3XK7CA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.769442942 (4h21m57.358787929s) [noise=true , address=104.36.82.37:4657 , piece_id=THIARAH55CX7JNZVQQPYI53EYMMXYIIKLA6OZALSIIIPMXKUERIQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.784083974 (4h21m57.373428961s) [noise=true , address=104.36.82.47:4543 , piece_id=ADMNLQJAAL3LBEJKVMRXPE3YD7LQWDNCZZ7NJMZ52ZRPD6F3XKJQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.789657258 (4h21m57.379002245s) [noise=true , address=104.36.82.9:4009 , piece_id=HXBANDR2AIXFHWYVE7Q5HRSZCD6MB63MS5CFIGSSYF27Z4NBDURQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.794090951 (4h21m57.383435938s) [noise=true , address=104.36.82.61:4381 , piece_id=FC7OY57PJSFEXISS7ATZLBKF5ROYJ46IMRLFJJN3JTZZAEESYPTA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.796193133 (4h21m57.38553812s) [noise=true , address=104.36.82.8:4974 , piece_id=FMUFZCB73EPXFNONS4T5HVYIR5LJONVVG2HDSMBUDZPPGKMGG2IQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.796206829 (4h21m57.385551816s) [seg_pos={0 0}, upload=4560]: Segment reached optimal threshold of 14 pieces. Aug 7 14:01:57.796312842 (4h21m57.385657819s) [noise=true , address=104.36.82.16:4328 , piece_id=OZXU7U6ITSX2TGNZMQEI5XXOP4IEIFVLD4JQJUJYXP26CKMXD7PQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796431358 (4h21m57.385776345s) [noise=true , address=104.36.82.58:4276 , piece_id=S365XPUFKRWCH2R73XRSITWS67PBAVJ5HWZRQQNXVLWY3DTJQE7Q, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796470935 (4h21m57.385815922s) [noise=true , address=104.36.82.15:4281 , piece_id=QXGWPJN2TRKVD3NL3D37FLSUIE5VAFEUIWASXLOMGAZQ6OUFJSDQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hDXeAm4tomp1gAkBNjM8x77V76yiruYQmPQ3BZEYPLpJzigo2): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796622314 (4h21m57.385967301s) [noise=true , address=104.36.82.59:4304 , piece_id=5HAYGO6W4DJUF4RLOIAFNLBEDLQ3RZTIUNQV7USEIOMXJFKJJZDQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG): protocol: read tcp 188.166.225.170:60770->104.36.82.59:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 14:01:57.796634237 (4h21m57.385979224s) [seg_pos={0 0}, upload=4560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 14:01:57.796719510 (4h21m57.386064497s) [upload=4560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 14:01:57.796723478 (4h21m57.386068455s) [upload=4560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 14:01:58.268989039 (4h21m57.858334026s) [upload=4560]: Done uploading segments... Aug 7 14:01:58.269006392 (4h21m57.858351379s) [upload=4560]: Upload finished. err: <nil> Duration: 6.0408176s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.874091392s FAIL (2025-08-07 14:03:37 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.874115157s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.823668246s FAIL (2025-08-07 14:03:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.823693144s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.723149017s FAIL (2025-08-07 14:02:55 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.723178314s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X..................X.........................X......XX.] duration: 5.773360253s FAIL (2025-08-07 14:02:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 379a1e9654a2e11db197cba6085d9ca1a53aa639aef84e2a9b0d218a2128f1ca != before 3d2fb1b55f8ebd735fbff3c20592472b8ca42b215f2e645245a8bfe10574650b ======LOGS====== finished. err: <nil> Aug 7 14:01:55.179966005 (4h21m54.769310993s) [upload=4560]: Starting upload Aug 7 14:01:55.180027874 (4h21m54.769372861s) [upload=4560]: Done starting upload Aug 7 14:01:55.180144266 (4h21m54.769489253s) [upload=4560]: Uploading segments... Aug 7 14:01:55.180380849 (4h21m54.769725836s) [upload=4560]: Got next segment. Inline: false Aug 7 14:01:55.180414263 (4h21m54.76975925s) [upload=4560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 14:01:55.180418821 (4h21m54.769763808s) [upload=4560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 14:01:55.696218378 (4h21m55.285563365s) [seg_pos={0 0}, upload=4560]: Begin upload segment... Aug 7 14:01:55.696261209 (4h21m55.285606196s) [seg_pos={0 0}, upload=4560]: Done begin upload segment. Aug 7 14:01:55.696285175 (4h21m55.285630162s) [noise=true , address=104.36.82.37:4657 , piece_id=THIARAH55CX7JNZVQQPYI53EYMMXYIIKLA6OZALSIIIPMXKUERIQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696421506 (4h21m55.285766503s) [noise=true , address=104.36.82.36:4648 , piece_id=ZNTL7IT3QKPJWLY7CK3XTMZFNIUQFORSKQ6NXYWBGLORRYD3J66A, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696700209 (4h21m55.286045186s) [noise=true , address=104.36.82.59:4304 , piece_id=5HAYGO6W4DJUF4RLOIAFNLBEDLQ3RZTIUNQV7USEIOMXJFKJJZDQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696898969 (4h21m55.286243956s) [noise=true , address=104.36.82.16:4328 , piece_id=OZXU7U6ITSX2TGNZMQEI5XXOP4IEIFVLD4JQJUJYXP26CKMXD7PQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696993150 (4h21m55.286338137s) [noise=true , address=64.49.13.155:4914 , piece_id=4WTEGNBHLDKT7FU4ZHBOXMCGCQEQZOECCDXZ36AIRBFXK7HJMEMA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697235754 (4h21m55.286580741s) [noise=true , address=104.36.82.47:4543 , piece_id=ADMNLQJAAL3LBEJKVMRXPE3YD7LQWDNCZZ7NJMZ52ZRPD6F3XKJQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697291010 (4h21m55.286635997s) [noise=true , address=45.91.116.11:4124 , piece_id=ZF2NWECU4ECYBK5LBM57EWNGVZK4GP5TRPHDJN3NCDM564R3GVLA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697532753 (4h21m55.28687774s) [noise=true , address=45.91.116.7:4015 , piece_id=UWCY5JCZXORDTW2O7452QPNUOW5G5WO3IFAOHQ4Y7VWCGKOTOJDA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697573731 (4h21m55.286918718s) [noise=true , address=104.36.82.34:4565 , piece_id=5554GUPBKMWBZLSYBSCVGT62VPHLAU6KQKJZ4D3RQGUGYBQTKSWQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697921356 (4h21m55.287266343s) [noise=true , address=104.36.82.58:4276 , piece_id=S365XPUFKRWCH2R73XRSITWS67PBAVJ5HWZRQQNXVLWY3DTJQE7Q, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697988324 (4h21m55.287333311s) [noise=true , address=104.36.82.61:4381 , piece_id=FC7OY57PJSFEXISS7ATZLBKF5ROYJ46IMRLFJJN3JTZZAEESYPTA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698150995 (4h21m55.287495982s) [noise=true , address=45.91.116.13:4184 , piece_id=ZLMZYZFI66EKCAAGOQFD5KFNIAH2GGLI2ZQAGS74QSDP726OJ2QQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698310741 (4h21m55.287655728s) [noise=true , address=104.36.82.9:4009 , piece_id=HXBANDR2AIXFHWYVE7Q5HRSZCD6MB63MS5CFIGSSYF27Z4NBDURQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698446340 (4h21m55.287791328s) [noise=true , address=104.36.82.62:4422 , piece_id=CUKWVUNPGDNPJ53IULXXROM7FMYTA5C3GNBIGYMHQHEMTRKJMPYA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698593622 (4h21m55.287938609s) [noise=true , address=104.36.82.23:4451 , piece_id=ZIEI4KSNE3ZUB2EAAEEMP3TEQU6JLU22EVXYD6HALCODCR6M2W7Q, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698671090 (4h21m55.288016077s) [noise=true , address=104.36.82.8:4974 , piece_id=FMUFZCB73EPXFNONS4T5HVYIR5LJONVVG2HDSMBUDZPPGKMGG2IQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698843219 (4h21m55.288188206s) [noise=true , address=104.36.82.15:4281 , piece_id=QXGWPJN2TRKVD3NL3D37FLSUIE5VAFEUIWASXLOMGAZQ6OUFJSDQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.699065424 (4h21m55.288410411s) [noise=true , address=104.36.82.12:4170 , piece_id=JNYFNJG2JMCIWGOWNH2KFLR4TAYUHBAG2PLVOWPFY3NPGS3XK7CA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:56.874315860 (4h21m56.463660847s) [upload=4560]: Next returned nil segment Aug 7 14:01:56.874383459 (4h21m56.463728446s) [upload=4560]: Waiting for error group managing segments... Aug 7 14:01:57.175871398 (4h21m56.765216385s) [noise=true , address=64.49.13.155:4914 , piece_id=4WTEGNBHLDKT7FU4ZHBOXMCGCQEQZOECCDXZ36AIRBFXK7HJMEMA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.621709858 (4h21m57.211054845s) [noise=true , address=104.36.82.23:4451 , piece_id=ZIEI4KSNE3ZUB2EAAEEMP3TEQU6JLU22EVXYD6HALCODCR6M2W7Q, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.634524196 (4h21m57.223869183s) [noise=true , address=104.36.82.36:4648 , piece_id=ZNTL7IT3QKPJWLY7CK3XTMZFNIUQFORSKQ6NXYWBGLORRYD3J66A, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.639033634 (4h21m57.228378621s) [noise=true , address=104.36.82.62:4422 , piece_id=CUKWVUNPGDNPJ53IULXXROM7FMYTA5C3GNBIGYMHQHEMTRKJMPYA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.649834548 (4h21m57.239179545s) [noise=true , address=104.36.82.34:4565 , piece_id=5554GUPBKMWBZLSYBSCVGT62VPHLAU6KQKJZ4D3RQGUGYBQTKSWQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.669716348 (4h21m57.259061335s) [noise=true , address=45.91.116.13:4184 , piece_id=ZLMZYZFI66EKCAAGOQFD5KFNIAH2GGLI2ZQAGS74QSDP726OJ2QQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.686125904 (4h21m57.275470891s) [noise=true , address=45.91.116.11:4124 , piece_id=ZF2NWECU4ECYBK5LBM57EWNGVZK4GP5TRPHDJN3NCDM564R3GVLA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.687869821 (4h21m57.277214808s) [noise=true , address=45.91.116.7:4015 , piece_id=UWCY5JCZXORDTW2O7452QPNUOW5G5WO3IFAOHQ4Y7VWCGKOTOJDA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.697242272 (4h21m57.286587259s) [noise=true , address=104.36.82.12:4170 , piece_id=JNYFNJG2JMCIWGOWNH2KFLR4TAYUHBAG2PLVOWPFY3NPGS3XK7CA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.769442942 (4h21m57.358787929s) [noise=true , address=104.36.82.37:4657 , piece_id=THIARAH55CX7JNZVQQPYI53EYMMXYIIKLA6OZALSIIIPMXKUERIQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.784083974 (4h21m57.373428961s) [noise=true , address=104.36.82.47:4543 , piece_id=ADMNLQJAAL3LBEJKVMRXPE3YD7LQWDNCZZ7NJMZ52ZRPD6F3XKJQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.789657258 (4h21m57.379002245s) [noise=true , address=104.36.82.9:4009 , piece_id=HXBANDR2AIXFHWYVE7Q5HRSZCD6MB63MS5CFIGSSYF27Z4NBDURQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.794090951 (4h21m57.383435938s) [noise=true , address=104.36.82.61:4381 , piece_id=FC7OY57PJSFEXISS7ATZLBKF5ROYJ46IMRLFJJN3JTZZAEESYPTA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.796193133 (4h21m57.38553812s) [noise=true , address=104.36.82.8:4974 , piece_id=FMUFZCB73EPXFNONS4T5HVYIR5LJONVVG2HDSMBUDZPPGKMGG2IQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.796206829 (4h21m57.385551816s) [seg_pos={0 0}, upload=4560]: Segment reached optimal threshold of 14 pieces. Aug 7 14:01:57.796312842 (4h21m57.385657819s) [noise=true , address=104.36.82.16:4328 , piece_id=OZXU7U6ITSX2TGNZMQEI5XXOP4IEIFVLD4JQJUJYXP26CKMXD7PQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796431358 (4h21m57.385776345s) [noise=true , address=104.36.82.58:4276 , piece_id=S365XPUFKRWCH2R73XRSITWS67PBAVJ5HWZRQQNXVLWY3DTJQE7Q, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796470935 (4h21m57.385815922s) [noise=true , address=104.36.82.15:4281 , piece_id=QXGWPJN2TRKVD3NL3D37FLSUIE5VAFEUIWASXLOMGAZQ6OUFJSDQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hDXeAm4tomp1gAkBNjM8x77V76yiruYQmPQ3BZEYPLpJzigo2): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796622314 (4h21m57.385967301s) [noise=true , address=104.36.82.59:4304 , piece_id=5HAYGO6W4DJUF4RLOIAFNLBEDLQ3RZTIUNQV7USEIOMXJFKJJZDQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG): protocol: read tcp 188.166.225.170:60770->104.36.82.59:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 14:01:57.796634237 (4h21m57.385979224s) [seg_pos={0 0}, upload=4560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 14:01:57.796719510 (4h21m57.386064497s) [upload=4560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 14:01:57.796723478 (4h21m57.386068455s) [upload=4560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 14:01:58.268989039 (4h21m57.858334026s) [upload=4560]: Done uploading segments... Aug 7 14:01:58.269006392 (4h21m57.858351379s) [upload=4560]: Upload finished. err: <nil> Duration: 5.773392586s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.051088874s FAIL (2025-08-07 14:02:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 2.051115795s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X..................X.........................X......XX] duration: 6.357051438s FAIL (2025-08-07 14:02:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 379a1e9654a2e11db197cba6085d9ca1a53aa639aef84e2a9b0d218a2128f1ca != before 3d2fb1b55f8ebd735fbff3c20592472b8ca42b215f2e645245a8bfe10574650b ======LOGS====== finished. err: <nil> Aug 7 14:01:55.179966005 (4h21m54.769310993s) [upload=4560]: Starting upload Aug 7 14:01:55.180027874 (4h21m54.769372861s) [upload=4560]: Done starting upload Aug 7 14:01:55.180144266 (4h21m54.769489253s) [upload=4560]: Uploading segments... Aug 7 14:01:55.180380849 (4h21m54.769725836s) [upload=4560]: Got next segment. Inline: false Aug 7 14:01:55.180414263 (4h21m54.76975925s) [upload=4560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 14:01:55.180418821 (4h21m54.769763808s) [upload=4560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 14:01:55.696218378 (4h21m55.285563365s) [seg_pos={0 0}, upload=4560]: Begin upload segment... Aug 7 14:01:55.696261209 (4h21m55.285606196s) [seg_pos={0 0}, upload=4560]: Done begin upload segment. Aug 7 14:01:55.696285175 (4h21m55.285630162s) [noise=true , address=104.36.82.37:4657 , piece_id=THIARAH55CX7JNZVQQPYI53EYMMXYIIKLA6OZALSIIIPMXKUERIQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696421506 (4h21m55.285766503s) [noise=true , address=104.36.82.36:4648 , piece_id=ZNTL7IT3QKPJWLY7CK3XTMZFNIUQFORSKQ6NXYWBGLORRYD3J66A, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696700209 (4h21m55.286045186s) [noise=true , address=104.36.82.59:4304 , piece_id=5HAYGO6W4DJUF4RLOIAFNLBEDLQ3RZTIUNQV7USEIOMXJFKJJZDQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696898969 (4h21m55.286243956s) [noise=true , address=104.36.82.16:4328 , piece_id=OZXU7U6ITSX2TGNZMQEI5XXOP4IEIFVLD4JQJUJYXP26CKMXD7PQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.696993150 (4h21m55.286338137s) [noise=true , address=64.49.13.155:4914 , piece_id=4WTEGNBHLDKT7FU4ZHBOXMCGCQEQZOECCDXZ36AIRBFXK7HJMEMA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697235754 (4h21m55.286580741s) [noise=true , address=104.36.82.47:4543 , piece_id=ADMNLQJAAL3LBEJKVMRXPE3YD7LQWDNCZZ7NJMZ52ZRPD6F3XKJQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697291010 (4h21m55.286635997s) [noise=true , address=45.91.116.11:4124 , piece_id=ZF2NWECU4ECYBK5LBM57EWNGVZK4GP5TRPHDJN3NCDM564R3GVLA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697532753 (4h21m55.28687774s) [noise=true , address=45.91.116.7:4015 , piece_id=UWCY5JCZXORDTW2O7452QPNUOW5G5WO3IFAOHQ4Y7VWCGKOTOJDA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697573731 (4h21m55.286918718s) [noise=true , address=104.36.82.34:4565 , piece_id=5554GUPBKMWBZLSYBSCVGT62VPHLAU6KQKJZ4D3RQGUGYBQTKSWQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697921356 (4h21m55.287266343s) [noise=true , address=104.36.82.58:4276 , piece_id=S365XPUFKRWCH2R73XRSITWS67PBAVJ5HWZRQQNXVLWY3DTJQE7Q, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.697988324 (4h21m55.287333311s) [noise=true , address=104.36.82.61:4381 , piece_id=FC7OY57PJSFEXISS7ATZLBKF5ROYJ46IMRLFJJN3JTZZAEESYPTA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698150995 (4h21m55.287495982s) [noise=true , address=45.91.116.13:4184 , piece_id=ZLMZYZFI66EKCAAGOQFD5KFNIAH2GGLI2ZQAGS74QSDP726OJ2QQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698310741 (4h21m55.287655728s) [noise=true , address=104.36.82.9:4009 , piece_id=HXBANDR2AIXFHWYVE7Q5HRSZCD6MB63MS5CFIGSSYF27Z4NBDURQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698446340 (4h21m55.287791328s) [noise=true , address=104.36.82.62:4422 , piece_id=CUKWVUNPGDNPJ53IULXXROM7FMYTA5C3GNBIGYMHQHEMTRKJMPYA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698593622 (4h21m55.287938609s) [noise=true , address=104.36.82.23:4451 , piece_id=ZIEI4KSNE3ZUB2EAAEEMP3TEQU6JLU22EVXYD6HALCODCR6M2W7Q, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698671090 (4h21m55.288016077s) [noise=true , address=104.36.82.8:4974 , piece_id=FMUFZCB73EPXFNONS4T5HVYIR5LJONVVG2HDSMBUDZPPGKMGG2IQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.698843219 (4h21m55.288188206s) [noise=true , address=104.36.82.15:4281 , piece_id=QXGWPJN2TRKVD3NL3D37FLSUIE5VAFEUIWASXLOMGAZQ6OUFJSDQ, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:55.699065424 (4h21m55.288410411s) [noise=true , address=104.36.82.12:4170 , piece_id=JNYFNJG2JMCIWGOWNH2KFLR4TAYUHBAG2PLVOWPFY3NPGS3XK7CA, seg_pos={0 0}, upload=4560]: Uploading piece... Aug 7 14:01:56.874315860 (4h21m56.463660847s) [upload=4560]: Next returned nil segment Aug 7 14:01:56.874383459 (4h21m56.463728446s) [upload=4560]: Waiting for error group managing segments... Aug 7 14:01:57.175871398 (4h21m56.765216385s) [noise=true , address=64.49.13.155:4914 , piece_id=4WTEGNBHLDKT7FU4ZHBOXMCGCQEQZOECCDXZ36AIRBFXK7HJMEMA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.621709858 (4h21m57.211054845s) [noise=true , address=104.36.82.23:4451 , piece_id=ZIEI4KSNE3ZUB2EAAEEMP3TEQU6JLU22EVXYD6HALCODCR6M2W7Q, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.634524196 (4h21m57.223869183s) [noise=true , address=104.36.82.36:4648 , piece_id=ZNTL7IT3QKPJWLY7CK3XTMZFNIUQFORSKQ6NXYWBGLORRYD3J66A, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.639033634 (4h21m57.228378621s) [noise=true , address=104.36.82.62:4422 , piece_id=CUKWVUNPGDNPJ53IULXXROM7FMYTA5C3GNBIGYMHQHEMTRKJMPYA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.649834548 (4h21m57.239179545s) [noise=true , address=104.36.82.34:4565 , piece_id=5554GUPBKMWBZLSYBSCVGT62VPHLAU6KQKJZ4D3RQGUGYBQTKSWQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.669716348 (4h21m57.259061335s) [noise=true , address=45.91.116.13:4184 , piece_id=ZLMZYZFI66EKCAAGOQFD5KFNIAH2GGLI2ZQAGS74QSDP726OJ2QQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.686125904 (4h21m57.275470891s) [noise=true , address=45.91.116.11:4124 , piece_id=ZF2NWECU4ECYBK5LBM57EWNGVZK4GP5TRPHDJN3NCDM564R3GVLA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.687869821 (4h21m57.277214808s) [noise=true , address=45.91.116.7:4015 , piece_id=UWCY5JCZXORDTW2O7452QPNUOW5G5WO3IFAOHQ4Y7VWCGKOTOJDA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.697242272 (4h21m57.286587259s) [noise=true , address=104.36.82.12:4170 , piece_id=JNYFNJG2JMCIWGOWNH2KFLR4TAYUHBAG2PLVOWPFY3NPGS3XK7CA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.769442942 (4h21m57.358787929s) [noise=true , address=104.36.82.37:4657 , piece_id=THIARAH55CX7JNZVQQPYI53EYMMXYIIKLA6OZALSIIIPMXKUERIQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.784083974 (4h21m57.373428961s) [noise=true , address=104.36.82.47:4543 , piece_id=ADMNLQJAAL3LBEJKVMRXPE3YD7LQWDNCZZ7NJMZ52ZRPD6F3XKJQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.789657258 (4h21m57.379002245s) [noise=true , address=104.36.82.9:4009 , piece_id=HXBANDR2AIXFHWYVE7Q5HRSZCD6MB63MS5CFIGSSYF27Z4NBDURQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.794090951 (4h21m57.383435938s) [noise=true , address=104.36.82.61:4381 , piece_id=FC7OY57PJSFEXISS7ATZLBKF5ROYJ46IMRLFJJN3JTZZAEESYPTA, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.796193133 (4h21m57.38553812s) [noise=true , address=104.36.82.8:4974 , piece_id=FMUFZCB73EPXFNONS4T5HVYIR5LJONVVG2HDSMBUDZPPGKMGG2IQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: <nil> Aug 7 14:01:57.796206829 (4h21m57.385551816s) [seg_pos={0 0}, upload=4560]: Segment reached optimal threshold of 14 pieces. Aug 7 14:01:57.796312842 (4h21m57.385657819s) [noise=true , address=104.36.82.16:4328 , piece_id=OZXU7U6ITSX2TGNZMQEI5XXOP4IEIFVLD4JQJUJYXP26CKMXD7PQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S8PSdkLyaiURWkPGhcABSh3481hoswzukT1NKvpTkSp75vKDJ): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796431358 (4h21m57.385776345s) [noise=true , address=104.36.82.58:4276 , piece_id=S365XPUFKRWCH2R73XRSITWS67PBAVJ5HWZRQQNXVLWY3DTJQE7Q, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796470935 (4h21m57.385815922s) [noise=true , address=104.36.82.15:4281 , piece_id=QXGWPJN2TRKVD3NL3D37FLSUIE5VAFEUIWASXLOMGAZQ6OUFJSDQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hDXeAm4tomp1gAkBNjM8x77V76yiruYQmPQ3BZEYPLpJzigo2): protocol: expected piece hash; context canceled; EOF Aug 7 14:01:57.796622314 (4h21m57.385967301s) [noise=true , address=104.36.82.59:4304 , piece_id=5HAYGO6W4DJUF4RLOIAFNLBEDLQ3RZTIUNQV7USEIOMXJFKJJZDQ, seg_pos={0 0}, upload=4560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG): protocol: read tcp 188.166.225.170:60770->104.36.82.59:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 14:01:57.796634237 (4h21m57.385979224s) [seg_pos={0 0}, upload=4560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 14:01:57.796719510 (4h21m57.386064497s) [upload=4560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 14:01:57.796723478 (4h21m57.386068455s) [upload=4560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 14:01:58.268989039 (4h21m57.858334026s) [upload=4560]: Done uploading segments... Aug 7 14:01:58.269006392 (4h21m57.858351379s) [upload=4560]: Upload finished. err: <nil> Duration: 6.357076164s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.122437041s FAIL (2025-08-07 14:01:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.12246308s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.083558247s FAIL (2025-08-07 14:01:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 2.083607962s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.769359137s FAIL (2025-08-07 14:01:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.769386839s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X..................X.........................X......X] duration: 3.970905357s FAIL (2025-08-07 14:00:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4534]: Done uploading segments... Aug 7 14:00:15.432960620 (4h20m15.022305607s) [upload=4534]: Upload finished. err: <nil> Aug 7 14:00:54.033660171 (4h20m53.623005158s) [upload=4544]: Starting upload Aug 7 14:00:54.033725517 (4h20m53.623070504s) [upload=4544]: Done starting upload Aug 7 14:00:54.033834776 (4h20m53.623179763s) [upload=4544]: Uploading segments... Aug 7 14:00:54.033963752 (4h20m53.623308749s) [upload=4544]: Got next segment. Inline: false Aug 7 14:00:54.034007786 (4h20m53.623352773s) [upload=4544]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 14:00:54.034016192 (4h20m53.623361179s) [upload=4544]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 14:00:54.481727166 (4h20m54.071072153s) [seg_pos={0 0}, upload=4544]: Begin upload segment... Aug 7 14:00:54.481782412 (4h20m54.071127409s) [seg_pos={0 0}, upload=4544]: Done begin upload segment. Aug 7 14:00:54.481844149 (4h20m54.071189136s) [noise=true , address=104.36.82.62:4421 , piece_id=TK5TLYRAQZQYWHDEOJLZY26PWYUMC5HTB5SFYZ5PO34NZJMV3XPQ, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.481851483 (4h20m54.07119647s) [noise=true , address=104.36.82.23:4472 , piece_id=GAIAAYABAQWG5AVVF2GK7Q5KHO4DNHKP7VFXKF2EFNVVJGQR7VQA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.481899065 (4h20m54.071244052s) [noise=true , address=104.36.82.35:4594 , piece_id=EHOHJ67R5YMOJWB5MTOUTMTAGBPM4IYBDJ2PYGB7XX6Y2YA6BHIA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.482209087 (4h20m54.071554074s) [noise=true , address=104.36.82.9:4021 , piece_id=FVGVABE3TB3PGSI6G5E3VY5GXUYE3ZBW5N5FIYLOZEQRI56YHNXQ, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.482261648 (4h20m54.071606635s) [noise=true , address=64.49.13.155:4913 , piece_id=O66NDEISJWPSIYWJXUXULXEW5CR4BEWZ3GKNBQGUIZBZ7URINRMA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.482507929 (4h20m54.071852916s) [noise=true , address=104.36.82.60:4364 , piece_id=QDRTWLQ7IV6RAFT3V5WLNW5C7MYZCIGBEZJI2SVKISF3ANICOR7A, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.482629261 (4h20m54.071974248s) [noise=true , address=104.36.82.44:4886 , piece_id=4QUFMORX2CUYZ3VD2M6VUWTKQVVVHRHXOC3KPXTQ37EMLOHYXTNA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.482766614 (4h20m54.072111601s) [noise=true , address=64.49.13.26:4946 , piece_id=ZB2JGA6BX7LRQNN3QFB3OQ2R2BPEC2UGZS7X2JTRYQRQFQFHZMJA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.482945156 (4h20m54.072290143s) [noise=true , address=104.36.82.55:4219 , piece_id=CT3UWUMRWVIX5IOBOBNZEZVDOBLD5VE5B2JHFMWXATAJCZ7CSI6A, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483039576 (4h20m54.072384563s) [noise=true , address=104.36.82.46:4490 , piece_id=3BFES2W6Q2IKKFSEUSX6LTKH7XISLAJLMWMQIU33XOZG243TAOFA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483281119 (4h20m54.072626106s) [noise=true , address=104.36.82.8:4987 , piece_id=YZPDVUNCBSZTFRBXK264C5YHE5LWW5TLPLJBIVGPH2I2OETJWPFQ, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483287651 (4h20m54.072632638s) [noise=true , address=104.36.82.38:4721 , piece_id=QN4VIZC7WSJZ2EBXJGUGRV73NGJH4ITCKNSUIQA3K4Q7D5V53UEQ, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483533331 (4h20m54.072878318s) [noise=true , address=45.91.116.9:4284 , piece_id=D4PMNWFODZ5XWKYZ66UJUUMJHPMITIQBFS52XBJZUQHXYXSPMK3A, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483614286 (4h20m54.072959283s) [noise=true , address=104.36.82.4:4814 , piece_id=DIS37VFK3QTLW2XLPTO33HQO6RKWHCALATPSYBJX4ZLI45VGKG4A, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483750927 (4h20m54.073095914s) [noise=true , address=104.36.82.61:4377 , piece_id=R35T2MNZED2E47JQ2KNJYCUHH26UD6PQQBCAK6IYUPK7MQSGWOTQ, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.483936803 (4h20m54.07328179s) [noise=true , address=104.36.82.58:4281 , piece_id=C463IMJ6GWPG4RGKIJKILSTCCRXYWSXWGJ52XH7MK7UTFCVKEAKQ, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.484336378 (4h20m54.073681375s) [noise=true , address=104.36.82.59:4307 , piece_id=DPLEAAS2ZD57FE3NY2XD44YUP37HUQUF7NCM4RX56BZA34AA63KA, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:54.484663944 (4h20m54.074008931s) [noise=true , address=104.36.82.22:4510 , piece_id=7QYAEH3QMLBI7CCN444CCMP4H4H24ZY2Z56UIJXQ4AOFQJA4O25Q, seg_pos={0 0}, upload=4544]: Uploading piece... Aug 7 14:00:55.624732718 (4h20m55.214077705s) [upload=4544]: Next returned nil segment Aug 7 14:00:55.624750101 (4h20m55.214095089s) [upload=4544]: Waiting for error group managing segments... Aug 7 14:00:55.929792230 (4h20m55.519137227s) [noise=true , address=64.49.13.155:4913 , piece_id=O66NDEISJWPSIYWJXUXULXEW5CR4BEWZ3GKNBQGUIZBZ7URINRMA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:55.985677764 (4h20m55.575022751s) [noise=true , address=64.49.13.26:4946 , piece_id=ZB2JGA6BX7LRQNN3QFB3OQ2R2BPEC2UGZS7X2JTRYQRQFQFHZMJA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.400414799 (4h20m55.989759786s) [noise=true , address=104.36.82.8:4987 , piece_id=YZPDVUNCBSZTFRBXK264C5YHE5LWW5TLPLJBIVGPH2I2OETJWPFQ, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.456882458 (4h20m56.046227445s) [noise=true , address=45.91.116.9:4284 , piece_id=D4PMNWFODZ5XWKYZ66UJUUMJHPMITIQBFS52XBJZUQHXYXSPMK3A, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.456958472 (4h20m56.04630346s) [noise=true , address=104.36.82.23:4472 , piece_id=GAIAAYABAQWG5AVVF2GK7Q5KHO4DNHKP7VFXKF2EFNVVJGQR7VQA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.460711854 (4h20m56.050056841s) [noise=true , address=104.36.82.9:4021 , piece_id=FVGVABE3TB3PGSI6G5E3VY5GXUYE3ZBW5N5FIYLOZEQRI56YHNXQ, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.465695569 (4h20m56.055040556s) [noise=true , address=104.36.82.4:4814 , piece_id=DIS37VFK3QTLW2XLPTO33HQO6RKWHCALATPSYBJX4ZLI45VGKG4A, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.477502379 (4h20m56.066847376s) [noise=true , address=104.36.82.22:4510 , piece_id=7QYAEH3QMLBI7CCN444CCMP4H4H24ZY2Z56UIJXQ4AOFQJA4O25Q, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.488053877 (4h20m56.077398864s) [noise=true , address=104.36.82.35:4594 , piece_id=EHOHJ67R5YMOJWB5MTOUTMTAGBPM4IYBDJ2PYGB7XX6Y2YA6BHIA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.520846829 (4h20m56.110191846s) [noise=true , address=104.36.82.38:4721 , piece_id=QN4VIZC7WSJZ2EBXJGUGRV73NGJH4ITCKNSUIQA3K4Q7D5V53UEQ, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.554045197 (4h20m56.143390184s) [noise=true , address=104.36.82.62:4421 , piece_id=TK5TLYRAQZQYWHDEOJLZY26PWYUMC5HTB5SFYZ5PO34NZJMV3XPQ, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.555521621 (4h20m56.144866608s) [noise=true , address=104.36.82.55:4219 , piece_id=CT3UWUMRWVIX5IOBOBNZEZVDOBLD5VE5B2JHFMWXATAJCZ7CSI6A, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.576569853 (4h20m56.16591484s) [noise=true , address=104.36.82.61:4377 , piece_id=R35T2MNZED2E47JQ2KNJYCUHH26UD6PQQBCAK6IYUPK7MQSGWOTQ, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.607068475 (4h20m56.196413472s) [noise=true , address=104.36.82.58:4281 , piece_id=C463IMJ6GWPG4RGKIJKILSTCCRXYWSXWGJ52XH7MK7UTFCVKEAKQ, seg_pos={0 0}, upload=4544]: Done uploading piece. err: <nil> Aug 7 14:00:56.607102429 (4h20m56.196447436s) [seg_pos={0 0}, upload=4544]: Segment reached optimal threshold of 14 pieces. Aug 7 14:00:56.607257306 (4h20m56.196602303s) [noise=true , address=104.36.82.59:4307 , piece_id=DPLEAAS2ZD57FE3NY2XD44YUP37HUQUF7NCM4RX56BZA34AA63KA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127F8oKF6GRv6Rdht4BCPt4G9UhVXeWVHKGJRP4WrrkcfDMmApT): protocol: expected piece hash; context canceled; EOF Aug 7 14:00:56.607363039 (4h20m56.196708036s) [noise=true , address=104.36.82.46:4490 , piece_id=3BFES2W6Q2IKKFSEUSX6LTKH7XISLAJLMWMQIU33XOZG243TAOFA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF Aug 7 14:00:56.607400360 (4h20m56.196745437s) [noise=true , address=104.36.82.44:4886 , piece_id=4QUFMORX2CUYZ3VD2M6VUWTKQVVVHRHXOC3KPXTQ37EMLOHYXTNA, seg_pos={0 0}, upload=4544]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 7 14:00:56.607448091 (4h20m56.196793078s) [noise=true , address=104.36.82.60:4364 , piece_id=QDRTWLQ7IV6RAFT3V5WLNW5C7MYZCIGBEZJI2SVKISF3ANICOR7A, seg_pos={0 0}, upload=4544]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S9DQZhV6SDULbA9WKndh12CpFih6xtNcpdwUApD9d2XThFdFk): protocol: expected piece hash; context canceled; EOF Aug 7 14:00:56.607495041 (4h20m56.196840028s) [seg_pos={0 0}, upload=4544]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 14:00:56.607573160 (4h20m56.196918147s) [upload=4544]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 14:00:56.607577308 (4h20m56.196922295s) [upload=4544]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 14:00:56.919583912 (4h20m56.508928899s) [upload=4544]: Done uploading segments... Aug 7 14:00:56.919602928 (4h20m56.508947915s) [upload=4544]: Upload finished. err: <nil> Duration: 3.9709329s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.043910138s FAIL (2025-08-07 14:00:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 2.043944244s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.818473678s FAIL (2025-08-07 14:00:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.818501481s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X..................X.........................X......] duration: 5.93550916s FAIL (2025-08-07 14:00:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 5.935559547s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.880453452s FAIL (2025-08-07 13:59:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.880482548s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.772194203s FAIL (2025-08-07 13:59:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.772222167s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.840573288s FAIL (2025-08-07 13:59:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.840596813s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X..................X.........................X.....] duration: 5.727847082s FAIL (2025-08-07 13:59:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 5.727871979s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.826849412s FAIL (2025-08-07 13:58:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.826870945s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X..................X.........................X....] duration: 5.804707479s FAIL (2025-08-07 13:58:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 5.804731134s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.860146226s FAIL (2025-08-07 13:58:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.860170833s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.778206277s FAIL (2025-08-07 13:57:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.778231656s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X..................X.........................X...] duration: 5.695746328s FAIL (2025-08-07 13:57:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 5.695774961s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.937275707s FAIL (2025-08-07 13:57:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.937299062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.868616739s FAIL (2025-08-07 13:57:30 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.868654291s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X..................X.........................X..] duration: 5.856604603s FAIL (2025-08-07 13:57:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 5.856629932s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.811427666s FAIL (2025-08-07 13:56:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.811453063s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.865167686s FAIL (2025-08-07 13:56:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.865194547s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.........................................................X.] duration: 1.869202305s FAIL (2025-08-07 13:56:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.869275404s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.695484547s FAIL (2025-08-07 13:56:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.695513532s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X..................X.........................X.] duration: 5.999148814s FAIL (2025-08-07 13:55:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 5.999180184s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.........................................................X] duration: 1.62647492s FAIL (2025-08-07 13:55:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== sh batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:48:15.162718610 (4h8m14.752063587s) [upload=4331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:48:15.894155246 (4h8m15.483500233s) [upload=4331]: Done uploading segments... Aug 7 13:48:15.894172539 (4h8m15.483517526s) [upload=4331]: Upload finished. err: <nil> Aug 7 13:49:23.904055619 (4h9m23.493400596s) [upload=4348]: Starting upload Aug 7 13:49:23.904094805 (4h9m23.493439792s) [upload=4348]: Done starting upload Aug 7 13:49:23.904150271 (4h9m23.493495258s) [upload=4348]: Uploading segments... Aug 7 13:49:23.904183995 (4h9m23.493528982s) [upload=4348]: Got next segment. Inline: true Aug 7 13:49:23.904187742 (4h9m23.493532729s) [upload=4348]: Waiting for error group managing segments... Aug 7 13:49:23.904233570 (4h9m23.493578597s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:23.904240233 (4h9m23.49358522s) [upload=4348]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:23.904242828 (4h9m23.493587815s) [upload=4348]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:24.191927938 (4h9m23.781272925s) [upload=4348]: Done uploading segments... Aug 7 13:49:24.191947214 (4h9m23.781292201s) [upload=4348]: Upload finished. err: <nil> Aug 7 13:49:41.320452088 (4h9m40.909797075s) [upload=4352]: Starting upload Aug 7 13:49:41.320497054 (4h9m40.909842041s) [upload=4352]: Done starting upload Aug 7 13:49:41.320543232 (4h9m40.909888219s) [upload=4352]: Uploading segments... Aug 7 13:49:41.320573390 (4h9m40.909918377s) [upload=4352]: Got next segment. Inline: true Aug 7 13:49:41.320577177 (4h9m40.909922164s) [upload=4352]: Waiting for error group managing segments... Aug 7 13:49:41.320609029 (4h9m40.909954016s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:49:41.320612876 (4h9m40.909957863s) [upload=4352]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:49:41.320615441 (4h9m40.909960428s) [upload=4352]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:49:41.608558223 (4h9m41.19790321s) [upload=4352]: Done uploading segments... Aug 7 13:49:41.608576398 (4h9m41.197921385s) [upload=4352]: Upload finished. err: <nil> Aug 7 13:50:55.115493599 (4h10m54.704838586s) [upload=4373]: Starting upload Aug 7 13:50:55.115578362 (4h10m54.704923349s) [upload=4373]: Done starting upload Aug 7 13:50:55.115836275 (4h10m54.705181262s) [upload=4373]: Uploading segments... Aug 7 13:50:55.115944111 (4h10m54.705289128s) [upload=4373]: Got next segment. Inline: true Aug 7 13:50:55.115952958 (4h10m54.705297995s) [upload=4373]: Waiting for error group managing segments... Aug 7 13:50:55.115995700 (4h10m54.705340687s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:50:55.115999738 (4h10m54.705344715s) [upload=4373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:50:55.116004006 (4h10m54.705348983s) [upload=4373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:50:55.640199205 (4h10m55.229544192s) [upload=4373]: Done uploading segments... Aug 7 13:50:55.640221969 (4h10m55.229566966s) [upload=4373]: Upload finished. err: <nil> Aug 7 13:51:52.372594179 (4h11m51.961939166s) [upload=4392]: Starting upload Aug 7 13:51:52.372651779 (4h11m51.961996766s) [upload=4392]: Done starting upload Aug 7 13:51:52.372707015 (4h11m51.962052002s) [upload=4392]: Uploading segments... Aug 7 13:51:52.372748114 (4h11m51.962093101s) [upload=4392]: Got next segment. Inline: true Aug 7 13:51:52.372753213 (4h11m51.962098201s) [upload=4392]: Waiting for error group managing segments... Aug 7 13:51:52.372785014 (4h11m51.962130001s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:51:52.372789042 (4h11m51.962134029s) [upload=4392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:51:52.372791216 (4h11m51.962136193s) [upload=4392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:51:52.651845393 (4h11m52.24119038s) [upload=4392]: Done uploading segments... Aug 7 13:51:52.651866815 (4h11m52.241211802s) [upload=4392]: Upload finished. err: <nil> Aug 7 13:52:31.144886892 (4h12m30.734231879s) [upload=4400]: Starting upload Aug 7 13:52:31.144947699 (4h12m30.734292686s) [upload=4400]: Done starting upload Aug 7 13:52:31.145030747 (4h12m30.734375734s) [upload=4400]: Uploading segments... Aug 7 13:52:31.145076044 (4h12m30.734421032s) [upload=4400]: Got next segment. Inline: true Aug 7 13:52:31.145079832 (4h12m30.734424809s) [upload=4400]: Waiting for error group managing segments... Aug 7 13:52:31.145118105 (4h12m30.734463092s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:52:31.145133514 (4h12m30.734478501s) [upload=4400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:52:31.145136139 (4h12m30.734481116s) [upload=4400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:52:31.431169519 (4h12m31.020514506s) [upload=4400]: Done uploading segments... Aug 7 13:52:31.431188195 (4h12m31.020533182s) [upload=4400]: Upload finished. err: <nil> Aug 7 13:53:08.777110543 (4h13m8.36645557s) [upload=4411]: Starting upload Aug 7 13:53:08.777204873 (4h13m8.36654989s) [upload=4411]: Done starting upload Aug 7 13:53:08.777292812 (4h13m8.366637799s) [upload=4411]: Uploading segments... Aug 7 13:53:08.777343308 (4h13m8.366688336s) [upload=4411]: Got next segment. Inline: true Aug 7 13:53:08.777350722 (4h13m8.36669578s) [upload=4411]: Waiting for error group managing segments... Aug 7 13:53:08.777411438 (4h13m8.366756475s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:53:08.777419303 (4h13m8.36676433s) [upload=4411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:53:08.777425184 (4h13m8.366770201s) [upload=4411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:53:09.061090294 (4h13m8.650435281s) [upload=4411]: Done uploading segments... Aug 7 13:53:09.061107496 (4h13m8.650452473s) [upload=4411]: Upload finished. err: <nil> Aug 7 13:54:13.127652051 (4h14m12.716997038s) [upload=4427]: Starting upload Aug 7 13:54:13.127716403 (4h14m12.71706138s) [upload=4427]: Done starting upload Aug 7 13:54:13.127800033 (4h14m12.71714501s) [upload=4427]: Uploading segments... Aug 7 13:54:13.127832376 (4h14m12.717177353s) [upload=4427]: Got next segment. Inline: true Aug 7 13:54:13.127836434 (4h14m12.717181421s) [upload=4427]: Waiting for error group managing segments... Aug 7 13:54:13.127894114 (4h14m12.717239101s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:13.127898973 (4h14m12.71724396s) [upload=4427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:13.127903532 (4h14m12.717248549s) [upload=4427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:13.463972574 (4h14m13.053317561s) [upload=4427]: Done uploading segments... Aug 7 13:54:13.463987873 (4h14m13.05333286s) [upload=4427]: Upload finished. err: <nil> Aug 7 13:54:50.834737674 (4h14m50.424082662s) [upload=4439]: Starting upload Aug 7 13:54:50.834787840 (4h14m50.424132827s) [upload=4439]: Done starting upload Aug 7 13:54:50.834966452 (4h14m50.424311439s) [upload=4439]: Uploading segments... Aug 7 13:54:50.834996349 (4h14m50.424341336s) [upload=4439]: Got next segment. Inline: true Aug 7 13:54:50.835000357 (4h14m50.424345344s) [upload=4439]: Waiting for error group managing segments... Aug 7 13:54:50.835031085 (4h14m50.424376072s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:54:50.835034802 (4h14m50.424379789s) [upload=4439]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:54:50.835037347 (4h14m50.424382324s) [upload=4439]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:54:51.236764008 (4h14m50.826108995s) [upload=4439]: Done uploading segments... Aug 7 13:54:51.236781783 (4h14m50.82612677s) [upload=4439]: Upload finished. err: <nil> Aug 7 13:55:12.852977367 (4h15m12.442322354s) [upload=4449]: Starting upload Aug 7 13:55:12.853067630 (4h15m12.442412617s) [upload=4449]: Done starting upload Aug 7 13:55:12.853125801 (4h15m12.442470788s) [upload=4449]: Uploading segments... Aug 7 13:55:12.853158114 (4h15m12.442503101s) [upload=4449]: Got next segment. Inline: true Aug 7 13:55:12.853162542 (4h15m12.442507519s) [upload=4449]: Waiting for error group managing segments... Aug 7 13:55:12.853197559 (4h15m12.442542536s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.853205774 (4h15m12.442550751s) [upload=4449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:12.853208359 (4h15m12.442553346s) [upload=4449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:13.125412310 (4h15m12.714757297s) [upload=4449]: Done uploading segments... Aug 7 13:55:13.125437198 (4h15m12.714782195s) [upload=4449]: Upload finished. err: <nil> Aug 7 13:55:47.628821506 (4h15m47.218166493s) [upload=4457]: Starting upload Aug 7 13:55:47.628864678 (4h15m47.218209655s) [upload=4457]: Done starting upload Aug 7 13:55:47.628934532 (4h15m47.218279519s) [upload=4457]: Uploading segments... Aug 7 13:55:47.628978316 (4h15m47.218323293s) [upload=4457]: Got next segment. Inline: true Aug 7 13:55:47.628993244 (4h15m47.218338231s) [upload=4457]: Waiting for error group managing segments... Aug 7 13:55:47.629028732 (4h15m47.218373719s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:47.629034243 (4h15m47.21837923s) [upload=4457]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:55:47.629036588 (4h15m47.218381575s) [upload=4457]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:47.927925053 (4h15m47.51727004s) [upload=4457]: Done uploading segments... Aug 7 13:55:47.927945021 (4h15m47.517290008s) [upload=4457]: Upload finished. err: <nil> Duration: 1.626503826s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.914787394s FAIL (2025-08-07 13:55:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.914817503s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.904273257s FAIL (2025-08-07 13:55:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.904296672s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X..................X.........................X] duration: 4.098565676s FAIL (2025-08-07 13:55:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== d finished. err: <nil> Aug 7 13:55:12.706500097 (4h15m12.295845084s) [upload=4448]: Starting upload Aug 7 13:55:12.706548739 (4h15m12.295893726s) [upload=4448]: Done starting upload Aug 7 13:55:12.706618743 (4h15m12.29596373s) [upload=4448]: Uploading segments... Aug 7 13:55:12.706754593 (4h15m12.29609958s) [upload=4448]: Got next segment. Inline: false Aug 7 13:55:12.706786945 (4h15m12.296131932s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:55:12.706791774 (4h15m12.296136761s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:55:13.166749359 (4h15m12.756094346s) [seg_pos={0 0}, upload=4448]: Begin upload segment... Aug 7 13:55:13.166799755 (4h15m12.756144742s) [seg_pos={0 0}, upload=4448]: Done begin upload segment. Aug 7 13:55:13.166825505 (4h15m12.756170492s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.166857696 (4h15m12.756202683s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167246490 (4h15m12.756591477s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167343536 (4h15m12.756688523s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167543538 (4h15m12.756888525s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.167780602 (4h15m12.757125589s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168081237 (4h15m12.757426224s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.168883733 (4h15m12.75822873s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169158447 (4h15m12.758503435s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169408447 (4h15m12.758753434s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.169661501 (4h15m12.759006488s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172086310 (4h15m12.761431297s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.172625110 (4h15m12.761970097s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.178950172 (4h15m12.768295159s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.180340172 (4h15m12.769685159s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.181620100 (4h15m12.770965087s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.182945577 (4h15m12.772290564s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:13.184273908 (4h15m12.773618895s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Uploading piece... Aug 7 13:55:14.670596410 (4h15m14.259941397s) [upload=4448]: Next returned nil segment Aug 7 13:55:14.670612810 (4h15m14.259957787s) [upload=4448]: Waiting for error group managing segments... Aug 7 13:55:15.082053169 (4h15m14.671398156s) [noise=true , address=45.91.116.6:4999 , piece_id=LWLKD55CZKWHPYCPPOYU3QU62BD65RCMYPCS2RGSAMSJIVFOFXVA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.112546551 (4h15m14.701891538s) [noise=true , address=104.36.82.60:4349 , piece_id=DB5S4XRJMD2ADQEFOJN7HXTFJGATW7ID3EEKZ2JYN7H73OTOZBFQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.133147997 (4h15m14.722492974s) [noise=true , address=45.91.116.11:4141 , piece_id=ZQVRFGZ5XFB6NDL2D5GRHU324N4W5FDPGZTNLTY3S5TJM6KCTNOA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.144135218 (4h15m14.733480286s) [noise=true , address=104.36.82.43:4851 , piece_id=2APVMYUQENGGMAXTUEI5WMTYKUZVZ5PZO77HAEXZ7NO6QAISLNEA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.154286551 (4h15m14.743631528s) [noise=true , address=45.91.116.8:4063 , piece_id=LTGVK633Z4KV5WAUR3LNAUKSOMIY6TJBPZIK36O7ZBUIBVFRLN4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.176329074 (4h15m14.765674051s) [noise=true , address=104.36.82.18:4385 , piece_id=XFLV4EP6773FM7AZ2HXRYQT5QLLE6X575OUGVYBQUJWCTV4MRSYQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.180250296 (4h15m14.769595323s) [noise=true , address=104.36.82.54:4165 , piece_id=B6TVUNAIBTUQEHV6N6PIBPNEYH3B5C7IST5POKUOXIF4LJW3TLMQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.183367741 (4h15m14.772712728s) [noise=true , address=104.36.82.3:4773 , piece_id=DYXMNVU3QLRNUXHNSKK3SXZSCLV6SKSELF4QSXI2WSHEDTPQUQLA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.196120861 (4h15m14.785465838s) [noise=true , address=45.91.116.13:4176 , piece_id=JNARAAPLDSGEQNPU5WKU3GEEKK3Z4OK5MKFOZOJNMHC2HWOAXGUA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.202791785 (4h15m14.792136772s) [noise=true , address=104.36.82.62:4421 , piece_id=QOW7YUSXTROPOECNYNN65XOGCHYW64K22AL6BZ7BHPFQN4VIMXGQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.215575252 (4h15m14.804920239s) [noise=true , address=104.36.82.59:4331 , piece_id=BI6OWI6XVZWQCK5JJKNS335HI6236Q4VPPACY3CVG6HZMQG3DLHQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.221435394 (4h15m14.810780382s) [noise=true , address=45.91.116.7:4036 , piece_id=TH4CRQVKR3YEQHKWRLYEWTL3BYNQI6OWFLUSI3UUOUQWP4OVX3UA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.240138409 (4h15m14.829483386s) [noise=true , address=104.36.82.19:4408 , piece_id=QZJE76VMFL3V3DXKK77TNRJULDG6IHCLP35HOSAACJELM4Q5C4LA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263285345 (4h15m14.852630332s) [noise=true , address=104.36.82.8:4979 , piece_id=OOH2G25XRMJHDFFMVOWPHGXBFQTAF6S47QHP3MN4U4MIVNXXJZ5Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: <nil> Aug 7 13:55:15.263312016 (4h15m14.852657003s) [seg_pos={0 0}, upload=4448]: Segment reached optimal threshold of 14 pieces. Aug 7 13:55:15.263402780 (4h15m14.852747757s) [noise=true , address=104.36.82.23:4454 , piece_id=GPKIDPKZ2CKKQMVI4XFH7PRUBXDYOA2TYRITNOSSMVDE24NLEAPQ, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263494946 (4h15m14.852839923s) [noise=true , address=104.36.82.9:4017 , piece_id=WPBDFGIGMLIANOZUOLLPZLEXSRJIKTRRMNWQBWM45ZZ5BZKRJLMA, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263554240 (4h15m14.852899227s) [noise=true , address=104.36.82.47:4541 , piece_id=GHILCMR46FVETL53OQPZQEPEFVEVMWRGMSKZZX7NUM3OD6ARLH4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC): protocol: expected piece hash; context canceled; EOF Aug 7 13:55:15.263661315 (4h15m14.853006302s) [noise=true , address=104.36.82.61:4399 , piece_id=7B4UW6IEM4AX5MRYEQIXQB3NTZMQQQISYHL7SB6H27RKJ32R6X4Q, seg_pos={0 0}, upload=4448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n3MKoczTv1AVSW5FJf79e1i7zSvVVePNzoGx3atmJ6SuDDZvr): protocol: read tcp 188.166.225.170:34890->104.36.82.61:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 13:55:15.263702323 (4h15m14.85304731s) [seg_pos={0 0}, upload=4448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:55:15.263779250 (4h15m14.853124237s) [upload=4448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:55:15.263783778 (4h15m14.853128756s) [upload=4448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:55:15.673015510 (4h15m15.262360497s) [upload=4448]: Done uploading segments... Aug 7 13:55:15.673033635 (4h15m15.262378622s) [upload=4448]: Upload finished. err: <nil> Duration: 4.098588268s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.828226146s FAIL (2025-08-07 13:55:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.828252295s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.100090857s FAIL (2025-08-07 13:54:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.100116636s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.686305828s FAIL (2025-08-07 13:54:50 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.686331046s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X..................X.........................] duration: 5.712327941s FAIL (2025-08-07 13:54:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.712354904s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.829829107s FAIL (2025-08-07 13:54:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.829847743s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X..................X........................] duration: 5.801985594s FAIL (2025-08-07 13:54:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.80200994s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.056510035s FAIL (2025-08-07 13:54:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.056530154s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X..................X.......................] duration: 5.625829459s FAIL (2025-08-07 13:53:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.625874596s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.72055705s FAIL (2025-08-07 13:53:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.720580255s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.96475033s FAIL (2025-08-07 13:53:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.964777442s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.................X..................X......................] duration: 5.87583098s FAIL (2025-08-07 13:52:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.875853743s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.869717941s FAIL (2025-08-07 13:52:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.869741406s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.712082274s FAIL (2025-08-07 13:52:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.712107363s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)