Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
3.923556534s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:09:52 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 3.923572084s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
16.465005959s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:09:28 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 16.465030306s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
3.937015183s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:08:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 3.937032286s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
16.898536577s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:07:48 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 16.898560503s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
5.911277084s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:06:59 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 5.911289848s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.002816568s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:06:53 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 1m0.002828141s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.159565712s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 06:01:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.159577485s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.093919732s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 06:00:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.093945302s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
3.946250633s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:59:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.946273778s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
3.963219949s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:58:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.963248213s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.026932935s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:57:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.02695626s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.113888328s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:56:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.113909169s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [X.....X.....................................................] duration:
4.134359066s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:55:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.134383333s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.X.....X....................................................] duration:
8.390676736s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:55:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 8.390697105s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..X.....X...................................................] duration:
7.04570598s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:54:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.045729637s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...X.....X..................................................] duration:
4.173035772s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:53:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.173063657s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....X.....X.................................................] duration:
4.073071216s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:53:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.073091265s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....X.....X................................................] duration:
8.686529439s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:53:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 8.686563594s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......X.....X...............................................] duration:
4.030519133s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:52:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.030540184s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.......X.....X..............................................] duration:
4.07904317s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:52:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.079067278s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [........X.....X.............................................] duration:
4.107824919s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:51:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.10784613s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.........X.....X............................................] duration:
12.058260377s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:51:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 12.058275997s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..........X.....X...........................................] duration:
36.545975577s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:50:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 36.545999802s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...........X.....X..........................................] duration:
7.084087158s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:49:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.084107708s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [............X.....X.........................................] duration:
4.069599131s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:48:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.069619109s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............X.....X........................................] duration:
7.500378492s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:47:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.500400094s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............X.....X.......................................] duration:
4.408999939s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:47:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.409018796s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...............X.....X......................................] duration:
4.064042955s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:46:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.064065006s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [................X.....X.....................................] duration:
4.104418763s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:45:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.104447509s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.................X.....X....................................] duration:
3.900449792s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:44:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.900463739s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..................X.....X...................................] duration:
3.909697963s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:43:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.909713373s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...................X.....X..................................] duration:
6.940379348s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:42:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.940399928s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....................X.....X.................................] duration:
5.627108754s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:41:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.627131918s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.....X................................] duration:
4.883372964s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:41:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.883395245s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......................X.....X...............................] duration:
4.355524392s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:40:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.355545092s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.....X..............................] duration:
4.148308897s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:39:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.148337662s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [........................X.....X.............................] duration:
3.978279736s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:38:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.978299313s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.....X............................] duration:
4.008680664s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:37:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.008702677s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..........................X.....X...........................] duration:
55.763380467s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:37:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 55.763402209s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...........................X.....X..........................] duration:
5.78098729s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:35:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.781011025s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [............................X.....X.........................] duration:
3.993865875s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:35:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.993887086s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............................X.....X........................] duration:
4.155798204s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:34:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.155820357s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............................X.....X.......................] duration:
4.110888706s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:33:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.110910408s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...............................X.....X......................] duration:
7.627079038s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:33:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.627100059s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [................................X.....X.....................] duration:
11.647293989s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:33:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 11.647306773s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.................................X.....X....................] duration:
45.760751659s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:33:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 45.760771156s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..................................X.....X...................] duration:
4.596986501s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:32:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.597017931s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...................................X.....X..................] duration:
4.103233933s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:30:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.103247288s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....................................X.....X.................] duration:
4.092045037s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:30:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.092069594s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X.....X................] duration:
6.952962308s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:29:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.95298315s
core
uplink
uk-select-1
uk1-placement-17
|