LinguaLibre

Difference between revisions of "Language codes systems used across LinguaLibre"

The current page gathers language codes systems used within or by LinguaLibre's community and tools. An emphasis is made to ease visibility on interoperability between resources.
 
(31 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
{{#Subtitle:The current page gathers language codes systems used within or by LinguaLibre's community and tools. An emphasis is made to ease visibility on interoperability between resources.}}
 +
 
== List of codes used ==
 
== List of codes used ==
−
Codes used by Lingualibre or useful for users and developers to know.
+
Codes systems used by Lingualibre or useful for users and developers to know. The <code>Short</code> column presents recommendations especially useful to follow when coding on open source code.
 
{| class="wikitable sortable"
 
{| class="wikitable sortable"
−
! Acronym || Name || Description || Reference page
+
! Short || English name || Description || Reference page
 
|-
 
|-
−
| iso639-3 || || Internation Stardard Organization's language code with 3 letters || [[:en:List of ISO 639-2 codes|List of ISO 639-2 codes]], column 2
+
| iso639-3 || ISO 639-3 || Internation Stardard Organization's language codes based on 3-letters codes. || Full:[[:en:Wikipedia:WikiProject_Languages/List_of_ISO_639-3_language_codes_(2019)|:en:List of ISO 639-3 language codes (2019)]]<br>Short:[[:en:List of ISO 639-2 codes]], column 2
 
|-
 
|-
−
| Qid || Wikidata Qid || Qid for each language on Wikidata.org, also this language page's title.  || [[LinguaLibre:List_of_languages]]
+
| qid || Wikidata Qid || Qid for each language on Wikidata.org, also this language page's title.  || [[LinguaLibre:List of languages]]
 
|-
 
|-
−
| Lid || Lingualibre Qid || Qid for each language imported to Lingualibre.org, also this language page's title. || [[LinguaLibre:List_of_languages]]  
+
| lid || Lingualibre Qid || Qid for each language imported to Lingualibre.org, also this language page's title. || [[LinguaLibre:List of languages]]  
 
|-
 
|-
−
| wikis || Wikipedia prefixes || Wikipedia.org subdomain prefixes for language-specific wikis || :en:[[:en:List of Wikipedias#Edition details|List of Wikipedias#Edition details]], column 3
+
| wiki || Wikipedia prefixes || Wikipedia.org subdomain prefixes for language-specific wikis || [[:en:List of Wikipedias#Edition details]], column 3<br> [[LinguaLibre:List of languages]]
 
|-
 
|-
−
| ? || ? || Language codes uses by Google's Corpuscrawler and UNILEX || [https://github.com/google/corpuscrawler Google/corpuscrawler]
+
| ietf || IETF BCP 47 language tag || ''Internet Engineering Task Force'' (IETF)'s created ''Best Current Pracices 47'' language codes on the basis of existing ISO 639, ISO 3166 and ISO 15924 standards.  || [[:en:IETF language tag]]
 
|}
 
|}
  
 
== Languages lists using language codes ==
 
== Languages lists using language codes ==
 +
Each of the project belows code their languages in some ways. Point here to a page of the project and how, there, they code their languages.
 
{| class="wikitable sortable"
 
{| class="wikitable sortable"
−
! Page || Quantity || iso639-3 || Qid || Lid || wikis || ?
+
! Page || Quantity || English || iso639-2 || iso639-3 || Qid || Lid || wikis || IETF
 
|-
 
|-
−
| [[LinguaLibre:List of languages]] || 100+ || yes || yes || yes || no  
+
| [[LinguaLibre:List of languages]] || 150+ || yes || no || yes || yes || yes || yes || no  
 
|-
 
|-
−
| :en:[[:en:List of Wikipedias#Edition details|Listof Wikipedias#Edition details]] || ~300 || no || no || no || yes || no
+
| :en:[[:en:List of Wikipedias#Edition details|Listof Wikipedias#Edition details]] || ~300 || yes || no ||  no || no || no || yes || no
 
|-
 
|-
−
| [https://github.com/google/corpuscrawler Google/corpuscrawler] || 1,001 || no || no || no || no || yes
+
| [https://github.com/google/corpuscrawler Google/corpuscrawler] || 1,001 || yes || no ||  no || no || no || no || yes
 +
|-
 +
| [https://github.com/unicode-org/unilex/ Unicode-org/UNILEX] || 1,000 || no || no ||  no || no || no || no || yes
 
|-  
 
|-  
−
| [https://github.com/unicode-org/unilex/ Unicode-org/UNILEX] || 1,000 || no || no || no || no || yes
+
| [https://github.com/hugolpz/Dragons_Bot/blob/main/languages.js Dragons_Bot/languages.js] || 1,000 || no || no || yes || no || no || no || yes
 
|-  
 
|-  
−
| [https://github.com/hugolpz/Dragons_Bot/ Dragons Bot] || 1,000 || no || no || no || yes || yes
+
| [https://opus.nlpl.eu/OpenSubtitles-v2018.php Open Subtitles 2018]<ref>Open subtitles 2018 languages list: `af,ar,bg,bn,br,bs,ca,cs,da,de,el,en,eo,es,et,eu,fa,fi,fr,gl,he,hi,hr,hu,hy,id,is,it,ja,ka,kk,ko,lt,lv,mk,ml,ms,nl,no,pl,pt,pt_br,ro,ru,si,sk,sl,sq,sr,sv,ta,te,th,tl,tr,uk,ur,vi,ze_en,ze_zh,zh_cn,zh_tw`</ref> || 63 || no || mostly<ref>https://opus.nlpl.eu/trac/wiki/DataFormats.html</ref> || no || no || no || no || no
 
|-  
 
|-  
−
| [https://opus.nlpl.eu/OpenSubtitles-v2018.php Open Subtitles 2018]<ref>Open subtitles 2018 languages: `af,ar,bg,bn,br,bs,ca,cs,da,de,el,en,eo,es,et,eu,fa,fi,fr,gl,he,hi,hr,hu,hy,id,is,it,ja,ka,kk,ko,lt,lv,mk,ml,ms,nl,no,pl,pt,pt_br,ro,ru,si,sk,sl,sq,sr,sv,ta,te,th,tl,tr,uk,ur,vi,ze_en,ze_zh,zh_cn,zh_tw`</ref> || 63 || ? || no || no || no || ?
+
| [https://opus.nlpl.eu/OpenSubtitles-v2018.php Tatoeba 2020]<ref>Tatoeba languages list: ab,acm,ady,af,afb,afh,aii,ain,ajp,akl,aln,am,an,ang,aoz,apc,ar,arq,ary,arz,as,ast,avk,awa,ayl,az,ba,bal,bar,be,ber,bg,bho,bjn,bm,bn,bo,br,brx,bs,bua,bvy,bzt,ca,cay,cbk,ce,ceb,ch,chg,chn,cho,chr,cjy,ckb,ckt,cmn,co,cpi,crh,crk,cs,csb,cv,cy,cycl,da,de,dng,drt,dsb,dtp,dv,dws,ee,egl,el,emx,en,enm,eo,es,et,eu,ext,fi,fj,fkv,fo,fr,frm,fro,frr,fuc,fur,fuv,fy,ga,gag,gan,gbm,gcf,gd,gil,gl,gn,gom,gos,got,grc,gsw,gu,gv,ha,hak,haw,hbo,he,hi,hif,hil,hnj,hoc,hr,hrx,hsb,hsn,ht,hu,hy,ia,iba,id,ie,ig,ii,ike,ilo,io,is,it,izh,ja,jam,jbo,jdt,jpa,jv,ka,kaa,kab,kam,kek,kha,kjh,kk,kl,km,kmr,kn,ko,koi,kpv,krc,krl,ksh,ku,kum,kw,kxi,ky,kzj,la,laa,lad,lb,ldn,lfn,lg,lij,liv,lkt,lld,lmo,ln,lo,lt,ltg,lut,lv,lzh,lzz,mad,mai,max,mdf,mfe,mg,mgm,mh,mhr,mi,mic,min,mk,ml,mn,mni,mnw,moh,mr,mt,mvv,mwl,mww,my,myv,na,nah,nan,nb,nch,nds,ngt,ngu,niu,nl,nlv,nn,nog,non,nov,npi,nst,nus,nv,ny,nys,oar,oc,ofs,ood,or,orv,os,osp,ota,otk,pa,pag,pal,pam,pap,pau,pcd,pdc,pes,phn,pi,pl,pms,pnb,ppl,prg,ps,pt,qu,quc,qya,rap,rif,rm,rn,ro,rom,ru,rue,rw,sa,sah,sc,scn,sco,sd,sdh,se,sg,sgs,shs,shy,si,sjn,sl,sm,sma,sn,so,sq,sr,stq,su,sux,sv,swg,swh,syc,ta,te,tet,tg,th,thv,ti,tig,tk,tl,tlh,tly,tmr,tmw,tn,to,toi,toki,tpi,tpw,tr,ts,tt,tts,tvl,ty,tyv,tzl,udm,ug,uk,umb,ur,uz,vec,vep,vi,vo,vro,wa,war,wo,wuu,xal,xh,xqa,yi,yo,yue,zlm,zsm,zu,zza`</ref> || 359 || ? || ? || ? || no || no || no || ?
 
|}
 
|}
 +
 +
== See also ==
 +
* [[Help:Langtags]]
 +
* [[:en:Language code#Common_schemes|Language code]] (en)
 +
* [[:meta:Language codes|Language codes]] (meta)
 +
* {{tl|To iso 639-2}}
 +
* {{tl|To iso 639-3}}
 +
* https://www.w3.org/International/articles/language-tags/
 +
 +
{{Helps}}
  
 
== References ==
 
== References ==
 
<references />
 
<references />

Latest revision as of 18:43, 14 September 2022


List of codes used

Codes systems used by Lingualibre or useful for users and developers to know. The Short column presents recommendations especially useful to follow when coding on open source code.

Short English name Description Reference page
iso639-3 ISO 639-3 Internation Stardard Organization's language codes based on 3-letters codes. Full::en:List of ISO 639-3 language codes (2019)
Short:en:List of ISO 639-2 codes, column 2
qid Wikidata Qid Qid for each language on Wikidata.org, also this language page's title. LinguaLibre:List of languages
lid Lingualibre Qid Qid for each language imported to Lingualibre.org, also this language page's title. LinguaLibre:List of languages
wiki Wikipedia prefixes Wikipedia.org subdomain prefixes for language-specific wikis en:List of Wikipedias#Edition details, column 3
LinguaLibre:List of languages
ietf IETF BCP 47 language tag Internet Engineering Task Force (IETF)'s created Best Current Pracices 47 language codes on the basis of existing ISO 639, ISO 3166 and ISO 15924 standards. en:IETF language tag

Languages lists using language codes

Each of the project belows code their languages in some ways. Point here to a page of the project and how, there, they code their languages.

Page Quantity English iso639-2 iso639-3 Qid Lid wikis IETF
LinguaLibre:List of languages 150+ yes no yes yes yes yes no
:en:Listof Wikipedias#Edition details ~300 yes no no no no yes no
Google/corpuscrawler 1,001 yes no no no no no yes
Unicode-org/UNILEX 1,000 no no no no no no yes
Dragons_Bot/languages.js 1,000 no no yes no no no yes
Open Subtitles 2018[1] 63 no mostly[2] no no no no no
Tatoeba 2020[3] 359 ? ? ? no no no ?

See also

Lingua Libre Help pages
General help pages Help:Interface • Help:Your first record • Help:Choosing a microphone • Help:Configure your microphone • Help:Translate • Help:Langtags • LinguaLibre:Language codes systems used across LinguaLibre • LinguaLibre:List of languages
Linguistic help pages Help:Add a new language • Help:Homographs • Help:List translation • Help:Ethics
Lists help pages Help:Create your own lists • Help:How to create a frequency list? • Help:Why wordlists matter? • Help:Swadesh lists • Help:Lists • Help:Create a new generator
Events, Outreach Lingualibre:Events • Lingualibre:Roles • Lingualibre:Workshops • Lingualibre:Hackathon • Lingualibre:Interested communities • Lingualibre:Events/2022 Public Relations Campaign • Lingualibre:Mailing • Lingualibre:Jargon • Lingualibre:Apps • Lingualibre:Citations • Service civique 2022-2023
Strategy Lingualibre 2022 Review (including outreach) • 2022-2023 Lingualibre wishlist • {{Wikimedia Language Diversity/Projects}} • Speakers map • Voices gender • Stats • Lingua Libre SignIt/2022 report • {{Grants}}


References

  1. ↑ Open subtitles 2018 languages list: `af,ar,bg,bn,br,bs,ca,cs,da,de,el,en,eo,es,et,eu,fa,fi,fr,gl,he,hi,hr,hu,hy,id,is,it,ja,ka,kk,ko,lt,lv,mk,ml,ms,nl,no,pl,pt,pt_br,ro,ru,si,sk,sl,sq,sr,sv,ta,te,th,tl,tr,uk,ur,vi,ze_en,ze_zh,zh_cn,zh_tw`
  2. ↑ https://opus.nlpl.eu/trac/wiki/DataFormats.html
  3. ↑ Tatoeba languages list: ab,acm,ady,af,afb,afh,aii,ain,ajp,akl,aln,am,an,ang,aoz,apc,ar,arq,ary,arz,as,ast,avk,awa,ayl,az,ba,bal,bar,be,ber,bg,bho,bjn,bm,bn,bo,br,brx,bs,bua,bvy,bzt,ca,cay,cbk,ce,ceb,ch,chg,chn,cho,chr,cjy,ckb,ckt,cmn,co,cpi,crh,crk,cs,csb,cv,cy,cycl,da,de,dng,drt,dsb,dtp,dv,dws,ee,egl,el,emx,en,enm,eo,es,et,eu,ext,fi,fj,fkv,fo,fr,frm,fro,frr,fuc,fur,fuv,fy,ga,gag,gan,gbm,gcf,gd,gil,gl,gn,gom,gos,got,grc,gsw,gu,gv,ha,hak,haw,hbo,he,hi,hif,hil,hnj,hoc,hr,hrx,hsb,hsn,ht,hu,hy,ia,iba,id,ie,ig,ii,ike,ilo,io,is,it,izh,ja,jam,jbo,jdt,jpa,jv,ka,kaa,kab,kam,kek,kha,kjh,kk,kl,km,kmr,kn,ko,koi,kpv,krc,krl,ksh,ku,kum,kw,kxi,ky,kzj,la,laa,lad,lb,ldn,lfn,lg,lij,liv,lkt,lld,lmo,ln,lo,lt,ltg,lut,lv,lzh,lzz,mad,mai,max,mdf,mfe,mg,mgm,mh,mhr,mi,mic,min,mk,ml,mn,mni,mnw,moh,mr,mt,mvv,mwl,mww,my,myv,na,nah,nan,nb,nch,nds,ngt,ngu,niu,nl,nlv,nn,nog,non,nov,npi,nst,nus,nv,ny,nys,oar,oc,ofs,ood,or,orv,os,osp,ota,otk,pa,pag,pal,pam,pap,pau,pcd,pdc,pes,phn,pi,pl,pms,pnb,ppl,prg,ps,pt,qu,quc,qya,rap,rif,rm,rn,ro,rom,ru,rue,rw,sa,sah,sc,scn,sco,sd,sdh,se,sg,sgs,shs,shy,si,sjn,sl,sm,sma,sn,so,sq,sr,stq,su,sux,sv,swg,swh,syc,ta,te,tet,tg,th,thv,ti,tig,tk,tl,tlh,tly,tmr,tmw,tn,to,toi,toki,tpi,tpw,tr,ts,tt,tts,tvl,ty,tyv,tzl,udm,ug,uk,umb,ur,uz,vec,vep,vi,vo,vro,wa,war,wo,wuu,xal,xh,xqa,yi,yo,yue,zlm,zsm,zu,zza`
Retrieved from "https://lingualibre.org/index.php?title=LinguaLibre:Language_codes_systems_used_across_LinguaLibre&oldid=791816"