LinguaLibre
Difference between revisions of "Language codes systems used across LinguaLibre"
The current page gathers language codes systems used within or by LinguaLibre's community and tools. An emphasis is made to ease visibility on interoperability between resources.
| Line 18: | Line 18: | ||
== Languages lists using language codes == | == Languages lists using language codes == | ||
| + | Each of the project belows code their languages in some ways. Point here to a page of the project and how, there, they code their languages. | ||
{| class="wikitable sortable" | {| class="wikitable sortable" | ||
! Page || Quantity || English || iso639-2 || iso639-3 || Qid || Lid || wikis || IETF | ! Page || Quantity || English || iso639-2 || iso639-3 || Qid || Lid || wikis || IETF | ||
Revision as of 17:18, 7 March 2021
List of codes used
Codes systems used by Lingualibre or useful for users and developers to know. The Short column presents recommendations especially useful to follow when coding on open source code.
| Short | English name | Description | Reference page |
|---|---|---|---|
| iso639-3 | ISO 639-3 | Internation Stardard Organization's language codes based on 3-letters codes. | Full::en:List of ISO 639-3 language codes (2019) Short:en:List of ISO 639-2 codes, column 2 |
| qid | Wikidata Qid | Qid for each language on Wikidata.org, also this language page's title. | LinguaLibre:List of languages |
| lid | Lingualibre Qid | Qid for each language imported to Lingualibre.org, also this language page's title. | LinguaLibre:List of languages |
| wiki | Wikipedia prefixes | Wikipedia.org subdomain prefixes for language-specific wikis | en:List of Wikipedias#Edition details, column 3 LinguaLibre:List of languages |
| ietf | IETF BCP 47 language tag | Internet Engineering Task Force (IETF)'s created Best Current Pracices 47 language codes on the basis of existing ISO 639, ISO 3166 and ISO 15924 standards. | en:IETF language tag |
Languages lists using language codes
Each of the project belows code their languages in some ways. Point here to a page of the project and how, there, they code their languages.
| Page | Quantity | English | iso639-2 | iso639-3 | Qid | Lid | wikis | IETF |
|---|---|---|---|---|---|---|---|---|
| LinguaLibre:List of languages | 100+ | yes | no | yes | yes | yes | yes | no |
| :en:Listof Wikipedias#Edition details | ~300 | yes | no | no | no | no | yes | no |
| Google/corpuscrawler | 1,001 | yes | no | no | no | no | no | yes |
| Unicode-org/UNILEX | 1,000 | no | no | no | no | no | no | yes |
| Dragons_Bot/languages.js | 1,000 | no | no | yes | no | no | no | yes |
| Open Subtitles 2018[1] | 63 | no | mostly[2] | no | no | no | no | no |
| Tatoeba 2020[3] | 359 | ? | ? | ? | no | no | no | ? |
See also
References
- ↑ Open subtitles 2018 languages list: `af,ar,bg,bn,br,bs,ca,cs,da,de,el,en,eo,es,et,eu,fa,fi,fr,gl,he,hi,hr,hu,hy,id,is,it,ja,ka,kk,ko,lt,lv,mk,ml,ms,nl,no,pl,pt,pt_br,ro,ru,si,sk,sl,sq,sr,sv,ta,te,th,tl,tr,uk,ur,vi,ze_en,ze_zh,zh_cn,zh_tw`
- ↑ https://opus.nlpl.eu/trac/wiki/DataFormats.html
- ↑ Tatoeba languages list: ab,acm,ady,af,afb,afh,aii,ain,ajp,akl,aln,am,an,ang,aoz,apc,ar,arq,ary,arz,as,ast,avk,awa,ayl,az,ba,bal,bar,be,ber,bg,bho,bjn,bm,bn,bo,br,brx,bs,bua,bvy,bzt,ca,cay,cbk,ce,ceb,ch,chg,chn,cho,chr,cjy,ckb,ckt,cmn,co,cpi,crh,crk,cs,csb,cv,cy,cycl,da,de,dng,drt,dsb,dtp,dv,dws,ee,egl,el,emx,en,enm,eo,es,et,eu,ext,fi,fj,fkv,fo,fr,frm,fro,frr,fuc,fur,fuv,fy,ga,gag,gan,gbm,gcf,gd,gil,gl,gn,gom,gos,got,grc,gsw,gu,gv,ha,hak,haw,hbo,he,hi,hif,hil,hnj,hoc,hr,hrx,hsb,hsn,ht,hu,hy,ia,iba,id,ie,ig,ii,ike,ilo,io,is,it,izh,ja,jam,jbo,jdt,jpa,jv,ka,kaa,kab,kam,kek,kha,kjh,kk,kl,km,kmr,kn,ko,koi,kpv,krc,krl,ksh,ku,kum,kw,kxi,ky,kzj,la,laa,lad,lb,ldn,lfn,lg,lij,liv,lkt,lld,lmo,ln,lo,lt,ltg,lut,lv,lzh,lzz,mad,mai,max,mdf,mfe,mg,mgm,mh,mhr,mi,mic,min,mk,ml,mn,mni,mnw,moh,mr,mt,mvv,mwl,mww,my,myv,na,nah,nan,nb,nch,nds,ngt,ngu,niu,nl,nlv,nn,nog,non,nov,npi,nst,nus,nv,ny,nys,oar,oc,ofs,ood,or,orv,os,osp,ota,otk,pa,pag,pal,pam,pap,pau,pcd,pdc,pes,phn,pi,pl,pms,pnb,ppl,prg,ps,pt,qu,quc,qya,rap,rif,rm,rn,ro,rom,ru,rue,rw,sa,sah,sc,scn,sco,sd,sdh,se,sg,sgs,shs,shy,si,sjn,sl,sm,sma,sn,so,sq,sr,stq,su,sux,sv,swg,swh,syc,ta,te,tet,tg,th,thv,ti,tig,tk,tl,tlh,tly,tmr,tmw,tn,to,toi,toki,tpi,tpw,tr,ts,tt,tts,tvl,ty,tyv,tzl,udm,ug,uk,umb,ur,uz,vec,vep,vi,vo,vro,wa,war,wo,wuu,xal,xh,xqa,yi,yo,yue,zlm,zsm,zu,zza`