Topics
Latest
AI
Amazon
Image Credits:Getty Images
Apps
Biotech & Health
mood
Image Credits:Getty Images
Cloud Computing
mercantilism
Crypto
Twelve Labs co-founder and CEO Jae Lee.Image Credits:Twelve Labs
Enterprise
EVs
Fintech
Twelve Labs builds multimodal video understanding models. Some answer questions, others perform searches — and more.Image Credits:Twelve Labs
fund raise
Gadgets
gage
Government & Policy
ironware
Layoffs
Media & Entertainment
Meta
Microsoft
Privacy
Robotics
Security
societal
outer space
Startups
TikTok
Transportation
speculation
More from TechCrunch
case
Startup Battlefield
StrictlyVC
Podcasts
Videos
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
AI model that realise video as well as text can unlock sinewy new app . At least , that ’s what Jae Lee , the co - laminitis ofTwelve Labs , think .
Granted , Lee ’s a little biased . Twelve Labs trains video recording - analyze model for a range of use lawsuit . But there may just be something to his assertion .
Using Twelve Labs ’ example , exploiter can seek through videos for specific moments , sum up clips , or necessitate questions like “ When did the person in the red shirt enroll the restaurant ? ” It ’s a potent hardening of capability — which is perhaps why the company has attracted big - name angel , including Nvidia , Samsung , and Intel .
Video search
To Lee , a data scientist by training , introductory search never made good sense for video . Keyword searches can rive up titles , tags , and descriptions , but ca n’t get at the actual content of clips .
“ Video is the fastest - get — and most data - intensive — intermediate , yet most organizations are n’t going to devote human resourcefulness to cull through all their picture archives , ” Lee told TechCrunch . “ Even if you tried manually tagging , it would n’t solve the issue . Finding a specific moment or angle in video can be like looking for a acerate leaf in a haystack . ”
After failing to notice a skillful solution , Lee recruited peers Aiden Lee , SJ Kim , Dave Chung , and Soyoung Lee to build one . That was the Book of Genesis of Twelve Labs , which trains fashion model to map textual matter to what ’s go on inside a video recording , include actions , objects , and scope sound .
Models like Google’sGeminican search through footage , and Microsoft and Amazon , among others , provide video analytics services to spot object in clips . But Lee argues that Twelve Labs ’ ware stand apart with their customization options , which allow customers tailor-make model using their own data .
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
“ companionship like OpenAI and Google are invest heavily in general - design multimodal model , ” Lee said , “ but these models are n’t optimized for telecasting . Our differentiation lies in being video recording - first from the first … We think video recording is deserving of our lone focus — it ’s not an add - on . ”
Devs can make apps on top of Twelve Labs modeling to search across video recording footage and more . The company ’s tech can drive things like ad interpolation , contentedness relief , and auto - generating high spot reels from clip .
When I speak with Leelast year , I asked about the electric potential for prejudice in Twelve Labs ’ models . It ’s a handsome risk of infection cistron . A 2021studyfound that training a video understanding model on clips of local word , which tends to cover crime in aracializedway , could cause the model to learn racist pattern .
Lee say at the fourth dimension that Twelve Labs was planning to release model - ethics - related benchmarks and datasets . The company still has n’t . In our late chat , Lee assured me that these tools are on the way and that Twelve Labs conducts bias tests on all of its models prior to releasing them .
“ We have n’t released formal prejudice benchmarks yet because we want to secure that they ’re meaningful , practical , and actionable , ” he said . “ Our overall goal is to develop benchmarks that not only hold us accountable , but also ready a standard in the industry … Until we ’ve amply accomplished this destination — and we have a team work on this — we ’re actively working to create AI that empower governance responsibly , respects people ’s civil liberties , and drives technological modification . ”
Lee added that Twelve Labs trains its models on a mix of public domain and licensed data , and does n’t source client data point for education .
Growth mode
video recording analysis remains core to what Twelve Labs does . But , in an effort to stay nimble , the company is also branching into surface area like “ any - to - any ” hunting and multimodal embeddings .
One of Twelve Labs ’ model , Marengo , can seek across images and audio in summation to video , and accept a reference work audio recording , prototype , or video clipping to help guide a search .
Elsewhere , the company offers an API , the Embed API , to create multimodal embeddings for videos , text , images , and audio file . Embeddings are numerical representations that capture the signification and relationship between dissimilar data full point , making them useful for applications programme like anomaly sensing .
Twelve Labs ’ growing product portfolio has helped the startup unafraid clients in the enterprise , media , and entertainment spaces . Two major better half are Databricks and Snowflake , both of which are building Twelve Labs tooling into their offering .
Databricks developed an desegregation that lets customer invoke Twelve Labs ’ embedding service from existing data grapevine . snow bunting , meanwhile , is creating connectors to Twelve Labs manakin in Cortex AI , its fully care AI service .
“ We currently have 30,000 - plus developer using our weapons platform , ranging from individuals experimenting to major endeavour integrating our technology into their workflow , ” Lee said . “ For illustration , we ’ve partner with municipalities for use case like actual - meter threat detection , enhancing emergency brake response prison term , and assisting in traffic management . ”
As a show of strategic support , both Databricks and Snowflake endow in Twelve Labs this calendar month through their several venture arms . SK Telecom and HubSpot Ventures join in , along with In - Q - Tel , an Arlington , Virginia - based non-profit-making VC that invests in startups bear out U.S. intelligence information capabilities .
The total raw investment came to $ 30 million , bringing Twelve Labs ’ total produce to $ 107.1 million . Lee sound out that the proceeds will be put toward mathematical product development and hiring .
“ We ’re in a very strong fiscal position , but saw an opportunity to intensify key strategic relationships with leadership who believe deeply in Twelve Labs , ” Lee order . “ We currently have 73 full - metre employees , and are plan significant investment in hire across technology , research , and customer - facing roles . ”
New hire
talk of hiring , Twelve Labs on Thursday declare that it ’s adding a president to its 100 - suite : Yoon Kim , former CTO of SK Telecom and a primal architect behind Apple ’s Siri . Yoon will also serve up as Twelve Labs ’ chief strategy officer , spearheading the startup ’s aggressive expansion plan .
“ While it ’s strange for a company of Twelve Labs ’ geezerhood and phase to charter a president , this move is a will to the demand we ’ve have , ” Lee say , adding that Yoon will split time between Twelve Labs ’ San Francisco HQ and its offices in Seoul . “ Yoon is the correct person to help us execute — he ’ll be instrumental in driving succeeding growth with cardinal acquisitions , expanding our global presence , and aligning our teams toward challenging goals . ”
Lee says that the target is to grow into new and adjacent vertical , like self-propelling and security , in the next few years . Considering In - Q - Tel ’s involvement , security ( and perhaps defence reaction piece of work ) seems like a shoe - in ; Lee would n’t sustain outright .
“ The investment from In - Q - Tel reflects the versatility and potential of our technology across many sector , including internal surety , ” Lee said . “ We ’re always open to explore opportunity where our engineering can have a convinced , meaningful , and responsible shock that aligns with our honorable road map . ”
