Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [......................X..X..............X....X.X......XX....] duration:
3.934833195s trace:
https://jaeger.dev.storj.io/trace/2a3e7f131cf1f7a0
FAIL (2026-04-12 15:51:49 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.93485618s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.......................X..X..............X....X.X......XX...] duration:
4.717706603s trace:
https://jaeger.dev.storj.io/trace/2a3e7f131cf1f7a0
FAIL (2026-04-12 15:51:12 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.717736841s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [........................X..X..............X....X.X......XX..] duration:
5.001467273s trace:
https://jaeger.dev.storj.io/trace/2a3e7f131cf1f7a0
FAIL (2026-04-12 15:50:00 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001527529s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.........................X..X..............X....X.X......XX.] duration:
5.001514785s trace:
https://jaeger.dev.storj.io/trace/2a3e7f131cf1f7a0
FAIL (2026-04-12 15:49:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001540846s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.359430413s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:48:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.359464289s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [..........................X..X..............X....X.X......XX] duration:
5.001435582s trace:
https://jaeger.dev.storj.io/trace/2a3e7f131cf1f7a0
FAIL (2026-04-12 15:48:27 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001450269s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
4.373659102s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:47:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.373687547s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
6 out of 60 is failed (threshold=6) [...........................X..X..............X....X.X......X] duration:
5.001753074s trace:
https://jaeger.dev.storj.io/trace/677147cb0ebb3cd6
FAIL (2026-04-12 15:47:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001775005s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.788169493s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:47:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.788197898s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [............................X..X..............X....X.X......] duration:
4.395945097s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:46:47 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.395969906s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.............................X..X..............X....X.X.....] duration:
3.982343589s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:46:27 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.982356164s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
4.840552312s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:46:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.840572443s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [..............................X..X..............X....X.X....] duration:
4.711607683s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:45:44 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.711633081s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.313354858s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:45:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 5.313380068s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [...............................X..X..............X....X.X...] duration:
4.293722567s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:44:45 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.293745402s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.267762341s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:44:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.267791367s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [................................X..X..............X....X.X..] duration:
4.316339154s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:43:35 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.316360446s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
4.674048381s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:43:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.674073239s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
4.177412315s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:42:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.177488471s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.................................X..X..............X....X.X.] duration:
4.776654839s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:42:47 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.776678045s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
4.291562646s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:41:59 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.291586562s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [..................................X..X..............X....X.X] duration:
5.000716953s trace:
https://jaeger.dev.storj.io/trace/b3933a4d20d22eb
FAIL (2026-04-12 15:41:33 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00073646s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
3.776422592s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:41:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 3.776449966s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...................................X..X..............X....X.] duration:
3.960051155s trace:
https://jaeger.dev.storj.io/trace/617d73a224bccaf3
FAIL (2026-04-12 15:40:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.960078838s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
4.836419015s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:40:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.836441868s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [....................................X..X..............X....X] duration:
5.001229887s trace:
https://jaeger.dev.storj.io/trace/617d73a224bccaf3
FAIL (2026-04-12 15:39:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001246046s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
4.228741651s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:39:24 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.228760447s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.....................................X..X..............X....] duration:
3.861393346s trace:
https://jaeger.dev.storj.io/trace/3b214873d5572d75
FAIL (2026-04-12 15:38:50 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.861416902s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
4.307487995s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:38:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.307513403s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [......................................X..X..............X...] duration:
4.232790141s trace:
https://jaeger.dev.storj.io/trace/3b214873d5572d75
FAIL (2026-04-12 15:38:22 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.232809999s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.......................................X..X..............X..] duration:
3.882310705s trace:
https://jaeger.dev.storj.io/trace/3b214873d5572d75
FAIL (2026-04-12 15:37:56 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.882345163s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
4.29613439s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:37:39 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.296158726s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [........................................X..X..............X.] duration:
4.795504019s trace:
https://jaeger.dev.storj.io/trace/3b214873d5572d75
FAIL (2026-04-12 15:37:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.795522325s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.........................................X..X..............X] duration:
5.000898031s trace:
https://jaeger.dev.storj.io/trace/3b214873d5572d75
FAIL (2026-04-12 15:36:59 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000913181s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
4.255341988s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:36:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.255366416s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
4.384340918s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:36:02 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.384371518s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........................................X..X..............] duration:
5.001424607s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:35:52 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001445518s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...........................................X..X.............] duration:
3.767572631s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:35:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.76760817s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
4.708469814s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:35:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.708496096s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
4.339875553s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:34:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.3399001s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [............................................X..X............] duration:
3.815565086s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:34:27 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.815601847s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
4.716803855s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:34:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.716826709s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.............................................X..X...........] duration:
3.885095141s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:33:51 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.885114088s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..............................................X..X..........] duration:
4.108498341s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:33:16 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.10852323s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
4.31770459s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:33:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.317726553s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...............................................X..X.........] duration:
3.838284544s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:33:00 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.838318669s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
4.172500305s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:32:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.172525844s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [................................................X..X........] duration:
4.140055746s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:32:16 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.140095663s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
4.736125328s trace:
https://jaeger.dev.storj.io/trace/6fe9f9d68d2fb238
FAIL (2026-04-12 15:31:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
dress=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.182046009 (119h16m18.001282033s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.183307873 (119h16m18.002543897s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.184574267 (119h16m18.003810291s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.185836313 (119h16m18.005072337s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.187304256 (119h16m18.00654037s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.188500495 (119h16m18.00773652s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.189661796 (119h16m18.00889782s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.190407676 (119h16m18.0096437s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.191832836 (119h16m18.01106886s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Uploading piece...
Apr 12 15:02:31.982892599 (119h16m18.802128623s) [upload=114392]: Next returned nil segment
Apr 12 15:02:31.982904202 (119h16m18.802140226s) [upload=114392]: Waiting for error group managing segments...
Apr 12 15:02:32.368146024 (119h16m19.187382058s) [noise=true , address=185.167.204.205:4014 , piece_id=6DY6SJZF3BKREMUU3LAJYG5Q4BZTHW5XCHOSS7QZZIOWNKOUWEDQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.369544603 (119h16m19.188780627s) [noise=true , address=185.167.204.205:4012 , piece_id=SHDSJW2LIFJ2TOKWKSW6F623H3LFN3E473G5YDE34DH6LSY67HSA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.372202162 (119h16m19.191438186s) [noise=true , address=185.167.204.206:4054 , piece_id=S5ZTF67XJSD2AYF53ILLJW3V4QHXQ2RIW64IR5FAB4QLQLRQBZ7Q, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.374919786 (119h16m19.19415581s) [noise=true , address=185.167.204.206:4068 , piece_id=WBQ2TSP6ZC5ZNY36XHYAOJHIZVGZXEOBVWU26VBTCPEHI44JKNTA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.378336279 (119h16m19.197572304s) [noise=true , address=185.167.204.205:4033 , piece_id=GPSRRCVI4AK5HFWDEY7RGP6DOM5PMZLR4RFDN2FVMLEZXWYJU7QQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382188595 (119h16m19.201424619s) [noise=true , address=185.167.204.206:4057 , piece_id=UGPCADO2VWZY5I5IO7AI76L567UGUSFK4JS225ISRUCNU7Z4LP4A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.382887724 (119h16m19.202123748s) [noise=true , address=185.167.204.205:4015 , piece_id=GTFM76FYUWVPHQFSVNDN6PEUK5UVO44RMFUGYZRS5F7MEP7MQCIQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.386601461 (119h16m19.205837485s) [noise=true , address=185.167.204.206:4045 , piece_id=BIWPNC57V6T7PPTZXW65DLJ6PNUL7PEAVIZZVKETF7GROEGJDVEA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.387915467 (119h16m19.207151491s) [noise=true , address=185.167.204.206:4071 , piece_id=HX2MK7TJXOB6RSYGQNHH23KH2D2S4R26SDXDGMJJOIKTWI5LJOKQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.389153798 (119h16m19.208389822s) [noise=true , address=185.167.204.205:4022 , piece_id=D6JHWCYVNOJTQ7LBOONAJIFRM5O72OATP4B3FHIMC5T643VJBTYA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.390760037 (119h16m19.209996061s) [noise=true , address=185.167.204.206:4050 , piece_id=KCZOSUZTBXAQOYJ2OKFGGQR5SC3BNP6RZVHEHKJRLJAWFVSS4O3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391233811 (119h16m19.210469836s) [noise=true , address=185.167.204.205:4007 , piece_id=KJNGPH64JFGVQMXOGDN5LL2T7LXAQK2VISZMEOYFRYWL7ELNTK3A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.391585260 (119h16m19.210821284s) [noise=true , address=185.167.204.205:4026 , piece_id=HNDC4TQTW2FG2OHFN64VHIX5CSDLJPYKJEIVWD27OAF5JEEEKBMQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.394116024 (119h16m19.213352058s) [noise=true , address=185.167.204.206:4038 , piece_id=NZ5HB4O76WPBN5N55BX6GPGEZECGNBJR2ZUF2PR6BM7VN44P3FUA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.396812939 (119h16m19.216048973s) [noise=true , address=185.167.204.206:4037 , piece_id=W4SFMX2IPEUGTGP7BSOK5MEN7QLLUI3D5EYDECRXQQUI4NL5DQOQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:32.406971875 (119h16m19.226207899s) [noise=true , address=185.167.204.206:4051 , piece_id=VBCP6DIJGOSHVILLP2SQDC3DUQXDJQCWUIERFEFIZZMYXXBE23ZQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.417594020 (119h16m20.236830054s) [noise=true , address=81.171.14.70:4033 , piece_id=EZO2GMYGDME5AUHNTYOV63KVBDMU4WIJMC2CDCGCPMRTXOFKPXQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.440786119 (119h16m20.260022143s) [noise=true , address=81.171.14.70:4026 , piece_id=3WXO773N5RISVCQYUOY7A3AIMU5HPL2XQKKDNFXJJJL7CJYAJELQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:33.725838220 (119h16m20.545074244s) [noise=true , address=81.171.14.70:4031 , piece_id=HLGISGHDTXZZVQ6DUFCNHI5OE34QSYGDFZFOZDFE6HS7XMDS4TQA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.462448441 (119h16m21.281684465s) [noise=true , address=89.149.201.147:4007 , piece_id=ZQEO7LPWQT3XXWVKGJYZY3QEPINFI5433L23FDBVXCWKVCP2GFXA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.488924547 (119h16m21.308160571s) [noise=true , address=89.149.201.147:4011 , piece_id=HTDMJR53D7AMGHL53C7TRSHWQS3WPWRXLUCFCMIQYN4LWVHRSESA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.555968007 (119h16m21.375204032s) [noise=true , address=89.149.201.147:4020 , piece_id=4P6JU2PWSXGME36IQH67YX3GKWVUTEFMPUS6XI4AWILUQH5LMFBQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.818964944 (119h16m21.638200968s) [noise=true , address=81.171.14.70:4041 , piece_id=SGAAGV5JU7BFETNN2RUNMCE26AK7HH4DDHKGIHTTJCFTO2K5IDIA, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:34.934084358 (119h16m21.753320382s) [noise=true , address=81.171.14.70:4030 , piece_id=7X5WVD5PXJZF6I4CG756RXIKBTFLZTWGZPFF752CULODUA2QNISQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:35.090526110 (119h16m21.909762134s) [noise=true , address=89.149.201.147:4021 , piece_id=YQETY2M7WU4HM57XBRWARWMXSHSRMCFJO47WRNU7WIQ3TILLN76A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217902244 (119h16m23.037138268s) [noise=true , address=89.149.201.147:4014 , piece_id=FEZ5DKYJQENZFAD27KAIXCKEAWHMM4IFFH7MHE4MNJN2BVS6G3UQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: <nil>
Apr 12 15:02:36.217927131 (119h16m23.037163155s) [seg_pos={0 0}, upload=114392]: Satisfied cohort requirements after 26 pieces
Apr 12 15:02:36.218149490 (119h16m23.037385514s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaeSxjXneBKoGKVrRM686sq386Bhn1YzzBFG9we8fHbcc9tbv): protocol: expected piece hash; context canceled; EOF
Apr 12 15:02:36.218157567 (119h16m23.037393591s) [noise=true , address=89.149.201.147:4023 , piece_id=QDKGGWB77T6B2ZB4QC6IM6X237GNC7EYMZAX3IJVW2R6SLKGW73A, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218165201 (119h16m23.037401215s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA): protocol: read tcp 188.166.225.170:36140->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 12 15:02:36.218171243 (119h16m23.037407267s) [noise=true , address=81.171.14.70:4035 , piece_id=AXCK2APNQL7YNNNXU2PFQ23Y2EECDDBTXV3YFW7HLCGAJIAUTCLQ, seg_pos={0 0}, upload=114392]: Long tail cancellation due to optimal threshold
Apr 12 15:02:36.218232531 (119h16m23.037468555s) [seg_pos={0 0}, upload=114392]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 12 15:02:36.218420784 (119h16m23.037656798s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 12 15:02:36.218425002 (119h16m23.037661016s) [upload=114392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 12 15:02:36.465312618 (119h16m23.284548663s) [upload=114392]: Done uploading segments...
Apr 12 15:02:36.465332226 (119h16m23.28456825s) [upload=114392]: Upload finished. err: <nil>
Duration: 4.736153082s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.................................................X..X.......] duration:
4.273728588s trace:
https://jaeger.dev.storj.io/trace/a7b8bfa5433de1f
FAIL (2026-04-12 15:31:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.273746442s
core
uplink
us-select-1
us1-placement-20
|