View source for Bot

{{LinguaLibre:Bot/intro}}
__TOC__
== Requests ==
Add your request below. Follow this template
<pre>
== Bot request for {language} witkionary ==
* '''Example pages (3):''' [[:ku:wikt:Apple]], [[:ku:wikt:Pomme]] - You can see best audio integration there.
* '''Target section:''' the audio file should be added at the end of the <code><nowiki>==Bilêvkirin==</nowiki></code> section, which means ...
* '''Local audio template(s) example(s):''' <nowiki>{{deng|en|en-us-apple.ogg|Deng (DYA)}}</nowiki> 
* '''Local audio template(s) explained:''' 
** {Deng} means "audio", and take the following parameters...
** <code>en</code> is the iso639-2 of the audio.
** <code>en-us-apple.ogg</code> is the filename
** <code>Deng (DYA)</code> means audio (deng) and USA (DYA) which is the local variant or accent.
* Request by: ~~~~ 
</pre>


== Bot request for ku.wiktionary ==
* '''Example pages (3):''' [[:ku:wikt:beran]], [[:ku:wikt:başûr]], [[:ku:wikt:keskesor]]- You can see best audio integration there.
* '''Target section:''' The audio file should be added at the end of the <code>=== Bilêvkirin ===</code> section, which means "Pronunciation". If there is no <code>=== Bilêvkirin ===</code> section on the page, please create one after the language section, that is <code>== {{<nowiki/>ziman|<lang code>}} ==</code>. If there is no language section, the audio file should not be added.
* '''Local audio template(s) example(s):''' <code>{{<nowiki/>deng|ku|LL-Q36368 (kur)-Mihemed Qers-keskesor.wav|Deng|dever=Qers}}</code>
* '''Local audio template(s) explained:''' 
** {Deng} is the template name which means "audio", and takes the following parameters...
** <code>ku</code> is the lang code from ISO 639-1 of the audio, ISO 639-3 and ISO 639-2 are also in use.
** <code>LL-Q36368 (kur)-Mihemed Qers-keskesor.wav</code> is the filename
** <code>Deng</code> means audio, should always be present.
** <code>|dever=</code> means place of origin, could be local variant or accent, country or city name. In the example "Qers" is the Kurdish name for the city [[:en:Kars]]. 
* Request by: [[User:Balyozxane|Balyozxane]] ([[User talk:Balyozxane|talk]]) 04:05, 22 February 2021 (UTC) 

:Here are two examples [https://ku.wiktionary.org/wiki/ba%C5%9F%C3%BBr], [https://ku.wiktionary.org/wiki/beran]. If there are multiple part of speech sections we still collect them all at the top of the page like this [https://ku.wiktionary.org/wiki/k%C3%AAm]. <code>|dever=</code> parameter should fetch the Kurdish names for places from Wikidata if possible. Lingua libre uses "kur" code for Kurdish, but we use "ku" and sometimes "kmr" on ku.wikt. Even when the language code is "kmr" in language section, the lang code in {{<nowiki/>deng|<lang code>}} should be "ku". I think that's all I can remember. Any questions? --[[User:Balyozxane|Balyozxane]] ([[User talk:Balyozxane|talk]]) 00:26, 21 February 2021 (UTC)
:: You can also take a look at this page [https://ku.wiktionary.org/w/index.php?title=%C5%9Eablon:deng/Lingualibre_bot&diff=next&oldid=4898008] for guidance.--[[User:Balyozxane|Balyozxane]] ([[User talk:Balyozxane|talk]]) 00:47, 21 February 2021 (UTC)
:::{{ping|Balyozxane}}, your last link is a diff, is it normal ? Also, can you reformat a bit your request so it follow the template above. You can also allow me to edit your text and I will happily do it. cc: [[user:Poslovitch]]. [[User:Yug|Yug]] ([[User talk:Yug|talk]]) 18:52, 21 February 2021 (UTC)
::::{{ping|Yug}} The last link was for example only to show there are other varients but the first two are the desired outcome from the LiLiBot. Feel free to correct my use of the template as free as you can. [[User:Balyozxane|Balyozxane]] ([[User talk:Balyozxane|talk]]) 04:05, 22 February 2021 (UTC)
:::::{{ping|Balyozxane}} Thanks! I'll get to work ASAP. I'll notify you once I'm ready to test the bot ;) --[[User:Poslovitch|Poslovitch]] ([[User talk:Poslovitch|talk]]) 13:19, 23 February 2021 (UTC)
::::::Thank you![[User:Balyozxane|Balyozxane]] ([[User talk:Balyozxane|talk]]) 08:00, 24 February 2021 (UTC)

== Connexion via Oauth and Bots for Unilex lists editing ==
{{ping|Olaf|Poslovitch}} Hello folks. I'am having some connection issues with WikiAPI (JS) code to connect to LinguaLibre. Is there some special thing to do to connect my bot to edit Lili ? As human using chrome, being connected to Commons alone doesn't connect you to LinguaLibre. We have to come here, click login, which sends a Oauth query (I guess), check my login status on Commons, then makes something so I'am loggued into both Commons and Lingualibre. I suspect some additional Oauth query is needed inside my bot. [[User:Yug|Yug]] ([[User talk:Yug|talk]]) 21:22, 1 March 2021 (UTC)
: Normally the login procedure here is very complicated: [[mw:OAuth/For_Developers]], and I've never managed to implement it, however if you use a bot account, you can create a password in [[Special:BotPasswords]], and then log in directly on Lingua Libre wiki without Commons. Alternatively you can use one of the [[mw:OAuth/For_Developers#Node.js|JS frameworks]] to log in. Finally, if you are logged in manually in the browser, the authorization proof should be in cookies, so the JS scripts in the browser should work fine. [[User:Olaf|Olaf]] ([[User talk:Olaf|talk]]) 21:19, 1 March 2021 (UTC)
::[[Special:BotPasswords/Dragons_Bot]]. Progress underway. Thank you.
::I see : 
<pre>Allowed IP ranges:
0.0.0.0/0
::/0</pre>
::Any explanation for this ? [[User:Dragons Bot|Dragons Bot]] ([[User talk:Dragons Bot|talk]]) 21:30, 1 March 2021 (UTC)
:::Starting to work. Situation unlocked. Thank [[user:Olaf|Olaf]]. [[User:Yug|Yug]] ([[User talk:Yug|talk]]) 22:08, 1 March 2021 (UTC)

== Lists: approach and limits ==
Dragons Bot script is ready to run. A test back is visible on [[Special:Contributions/Dragons_Bot]]. 

I propose the following ranges of words for lists creations :
<pre>
var ranges = [
	[ '00001', '00200' ],  // 1) 'List:Ibo/Most_used_words,_UNILEX_1:_words_00001_to_00200'
	[ '00201', '01000' ],  // 2) 'List:Ibo/Most_used_words,_UNILEX_2:_words_00201_to_01000'
	[ '01001', '02000' ],  // …
	[ '02001', '05000' ],  // 4) <←——— 1st threshold
	[ '05001', '10000' ],  // …
	[ '10001', '15000' ],  // 
	[ '15001', '20000' ],  // 
	[ '20001', '25000' ],  // 
	[ '25001', '30000' ],  // 9) <←——— 2nd threshold
	[ '30001', '35000' ],  // 
	[ '35001', '40000' ],  // 
	[ '40001', '45000' ],  // 
	[ '45001', '50000' ]   // 
];</pre>

I willfully create a '''smooth ramp approach''' to onboard new comers. I tested, 200 is a nice balance while we start. It is gently ambitious and about 10 mins works. It typically the kind of list-size I was looking for demoing in IRL events, with new users. Then can do just 20 if they wish. But the length alone, 200, encourage to flow it forward and to try out the Lingualibre productive flow which appears after 20~30 words but requires 50 words to "see the power" of LinguaLibre. 

As for the depth, I first though of a deal :
<pre>
// `corpus-limit`: 
//  - default: x = 5000; 
//  - active: x = 30000
//  - rule: if recordings > 2000 according to https://lingualibre.org/wiki/LinguaLibre:Stats, then `active`.
</pre>

With that rule, our 17 most active languages get 30,000 words via 9 files. All others get 5,000 words via 4 files.

But after some though I'am wondering if this 5,000 first limit is too small. It allows good on-boarding, but then nothing. Waiting still a bit. [[User:Yug|Yug]] ([[User talk:Yug|talk]]) 22:22, 4 March 2021 (UTC)
:{{ping|Pamputt}}, I derivated from the IETF's column on the right a new `iso639-3` column on the left. These `iso639-3` will provide the <code>Iso</code> in <code>List:{Iso}/{Title}{range}</code>. But I often didn't what was the `iso639-3` versions so I kept the IETF tag (ceux avec les <code>-</code>). Could you review [https://github.com/hugolpz/Dragons_Bot/blob/main/languages.js languages.js] and share with me possible corrective indications ? Or is it ok if I use those ? (I don't think so, the record wizard will have difficulties finding them) [[User:Yug|Yug]] ([[User talk:Yug|talk]]) 22:28, 4 March 2021 (UTC)
::About two letters code, you can find the "equivalent" ISO 639-3 code using this [[w:en:List_of_ISO_639-1_codes|Wikipedia page]]. For example, "ae-Latn" corresponds to "ave" on LinguaLibre. [[User:Pamputt|Pamputt]] ([[User talk:Pamputt|talk]]) 06:46, 5 March 2021 (UTC)
:::Yes, I need to be sure I'am converting correctly from composite IETF into current conventions (iso639-3 right?). Thanks for the lead on `ave`, I will check those. I gathered below the list of items I'am confused by. 
<pre>
	{ 'iso639-3':'ave', file:'ae-Latn' },
	{ 'iso639-3':'', file:'be-tarask' },
	{ 'iso639-3':'', file:'blt-Latn' },
	{ 'iso639-3':'', file:'ca-valencia' },
	{ 'iso639-3':'', file:'ctd-Latn' },
	{ 'iso639-3':'', file:'el-Latn-u-sd-it75' },
	{ 'iso639-3':'', file:'gsw-u-sd-chag' },
	{ 'iso639-3':'', file:'gsw-u-sd-chbe' },
	{ 'iso639-3':'', file:'gsw-u-sd-chfr' },
	{ 'iso639-3':'', file:'kab-Arab' },
	{ 'iso639-3':'', file:'kab-Tfng' },
	{ 'iso639-3':'', file:'rm-puter' },
	{ 'iso639-3':'', file:'rm-rumgr' },
	{ 'iso639-3':'', file:'rm-surmiran' },
	{ 'iso639-3':'', file:'rm-sursilv' },
	{ 'iso639-3':'', file:'rm-sutsilv' },
	{ 'iso639-3':'', file:'rm-vallader' },
	{ 'iso639-3':'', file:'sr-Latn' },
	{ 'iso639-3':'', file:'vec-u-sd-itpd' },
	{ 'iso639-3':'', file:'vec-u-sd-itts' },
	{ 'iso639-3':'', file:'vec-u-sd-itvr' },
</pre>
:::Do we have a naming convention for cases like <code>gsw</code>, <code>rm</code> and <code>vec</code> '''which has several sub elements each''' ? Should I do <code>List:{gsw}/u-sd-chag/{title}</code> ? I will return here later to complete all those I can. [[User:Yug|Yug]] ([[User talk:Yug|talk]])
::::What I know is Lingua Libre uses ISO 639-3 to identify the language in the lists. So we should use pure ISO 639-3 to name the lists. Let us talk about <code>rm</code> for example, the ISO 639-3 code is "roh". The text after the hyphen is used to discriminate the dialects. On LinguaLibre, we can create list for a given dialect, but it should be named such as "List:Roh/Puter-namelist" or maybe "List:Roh/Puter/Namelist". I have not checked what is the behaviour of the list taht would be named following the last proposal. 
:::: The same remark applied for IETF code such as "kab-Arab". It that case, "Arab" is about the script. So we could name the list such as "List:Kab/Arab/namelist" for example. [[User:Pamputt|Pamputt]] ([[User talk:Pamputt|talk]]) 08:59, 5 March 2021 (UTC)
::::: It's a more general question - how should the resulting recording files look like? For example <code>be-tarask</code> is a standard Belarussian but written with Latin script instead of Cirillic. Still most Wiktionaries considers it a separate language (example: [[wikt:fr:Catégorie:biélorusse_(tarashkevitsa)]]). If the LiLi bot is supposed to attach the recordings properly, the language should have a separate ISO code in LiLi. You can't just put it as a version of <code>bel</code>. But LiLi has only one Belarussian language code defined. 
::::: <code>sr-latin</code> is a Latinized version of Serbian, but the French Wiktionary puts it in the same bag with Cyrillic: [[:wikt:fr:Catégorie:serbe]], in the Polish Wiktionary we allow only the Cyrillic script in Serbian, in English Wiktionary everything is together with Croatian as the Serbo-Croatian language. Total mess. A few other codes are also different script versions of standard languages. 
::::: <code>gsw-*</code>, on the other hand, are various dialects of Swiss German. I believe they all are treated in Lingua Libre and Wiktionaries as dialects of German (<code>deu</code>). Perhaps code <code>gsw</code> could also be created here, but it isn't.
::::: <code>rm-*</code> are dialects of the Romansh language. LiLi treats them as one language <code>roh</code>. 
::::: In general, if we want to have rare languages on board, they should be defined here first. It's not enough to make a list if you can't select proper language while recording. Maybe you should import only lists for the languages defined in LiLi? [[User:Olaf|Olaf]] ([[User talk:Olaf|talk]]) 09:26, 5 March 2021 (UTC)
:::::: EDIT CONFLICTWhat I know is Lingua Libre uses ISO 639-3 to identify the language in the lists. So we should use pure ISO 639-3 to name the lists. Let us talk about <code>rm</code> for example, the ISO 639-3 code is "roh". The text after the hyphen is used to discriminate the dialects. On LinguaLibre, we can create list for a given dialect, but it should be named such as "List:Roh/Puter-namelist" or maybe "List:Roh/Puter/Namelist". I have not checked what is the behaviour of the list taht would be named following the last proposal. 
:::::: The same remark applied for IETF code such as "kab-Arab". It that case, "Arab" is about the script. So we could name the list such as "List:Kab/Arab/namelist" for example. [[User:Pamputt|Pamputt]] ([[User talk:Pamputt|talk]]) 09:43, 5 March 2021 (UTC)