Update README.md

must use crontab -e

.gitignore

@@ -6,12 +6,12 @@
 /botconf*
 /botacl*
 /botown*
-ˆ.jssh
+*.jssh
 *.save
 *.log
 *.swp
 *.swo
-/log/*
+/logs/
 /JSON.sh/
 /data-bot-bash/
 /DIST/

.gitmodules

@@ -1,3 +0,0 @@
-[submodule "JSON.sh"]
-	path = JSON.sh
-	url = http://github.com/dominictarr/JSON.sh

README.html

@@ -88,15 +88,15 @@ Bashbot - A Telegram bot written in bash.
 </h2>
 Written by Drew (@topkecleon) and Kay M (@gnadelwartz).
 
-<p>Contributions by Daniil Gentili (@danogentili), JuanPotato, BigNerd95, TiagoDanin, and iicc1.</p>
+<p>Contributions by Daniil Gentili (@danog), JuanPotato, BigNerd95, TiagoDanin, iicc1 and dcoomber.</p>
 <p>Released to the public domain wherever applicable. Elsewhere, consider it released under the <a href="http://www.wtfpl.net/txt/copying/">WTFPLv2</a>.</p>
 <p>Linted by <a href="https://github.com/koalaman/shellcheck">#ShellCheck</a></p>
 <h2>Prerequisites</h2>
-<p>Uses <a href="http://github.com/dominictarr/JSON.sh">JSON.sh</a> and the magic of sed.</p>
-<p>Even bashbot is written in bash, it depends on commands typically available in a Unix/Linux Environment. More concrete on the common commands provided by recent versions of <a href="https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands">coreutils</a>, <a href="https://en.wikipedia.org/wiki/BusyBox#Commands">busybox</a> or <a href="https://landley.net/toybox/help.html">toybox</a>, see <a href="doc/7_develop.md#common-commands">Developer Notes</a></p>
-<p><strong>Note for MacOS and BSD Users:</strong> As bashbot heavily uses modern bash and (gnu) grep/sed features, bashbot will not run without installing additional software, see <a href="doc/0_install.md">Install Bashbot</a></p>
-<p><strong>Note for embedded systems:</strong> busybox or toybox ONLY is not sufficient, you need a to install a "real" bash, see also <a href="doc/0_install.md">Install Bashbot</a></p>
-<p>Bashbot <a href="https://github.com/topkecleon/telegram-bot-bash">Documentation</a> and <a href="https://github.com/topkecleon/telegram-bot-bash/releases">Downloads</a> are available on www.github.com</p>
+<p>Uses <a href="http://github.com/dominictarr/JSON.sh">JSON.sh</a>/<a href="https://github.com/step-/JSON.awk">JSON.awk</a> and the magic of sed.</p>
+<p>Bashbot is written in bash. It depends on commands typically available in a Linux/Unix Environment. For more information on commands provided by recent versions of <a href="https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands">coreutils</a>, <a href="https://en.wikipedia.org/wiki/BusyBox#Commands">busybox</a> or <a href="https://landley.net/toybox/help.html">toybox</a>, see <a href="doc/7_develop.md#common-commands">Developer Notes</a>.</p>
+<p><strong>Note for MacOS and BSD Users:</strong> Bashbot will not run without installing additional software as it uses modern bash and (gnu) grep/sed features. See <a href="doc/0_install.md">Install Bashbot</a>.</p>
+<p><strong>Note for embedded systems:</strong> You need to install a "real" bash as the vanilla installation of busybox or toybox is not sufficient. See <a href="doc/0_install.md">Install Bashbot</a>.</p>
+<p>Bashbot <a href="https://github.com/topkecleon/telegram-bot-bash">Documentation</a> and <a href="https://github.com/topkecleon/telegram-bot-bash/releases">Downloads</a> are available on <a href="https://www.github.com">www.github.com</a>.</p>
 <h2>Documentation</h2>
 <ul>
 <li><a href="https://core.telegram.org/bots">Introduction to Telegram Bots</a></li>
@@ -153,126 +153,116 @@ Written by Drew (@topkecleon) and Kay M (@gnadelwartz).
 <li>Setup your environment</li>
 <li>Bashbot test suite</li>
 </ul></li>
-<li><a href="examples/README.md">Examples Dir</a></li>
+<li><a href="examples">Examples Directory</a></li>
+<li><a href="examples/webhook">Webhook Example</a></li>
 </ul>
-<h3>Your really first bashbot in a nutshell</h3>
-<p>To install and run bashbot you need access to a linux/unix command line. If you don't know how to get access to a linux/unix command line you should stop reading here :-(</p>
-<p>In addition you need a <a href="https://telegram.org">Telegram client</a> and a mobile phone to <a href="https://telegramguide.com/create-a-telegram-account/">register an account</a>. If you don't want to register for Telegram you should stop reading here ;-)</p>
-<p>After you're registered to Telegram send a message to <a href="https://telegram.me/botfather">@botfather</a>, <a href="doc/1_firstbot.md">create a new Telegram Bot token</a> and write it down. You need the token to install the bot.</p>
-<p>Now open a terminal and check if bash is installed:</p>
-<pre><code>which bash &amp;&amp; echo &quot;bash seems available...&quot;
-</code></pre>
-<p>Create a new directory, change to it: <code>mkdir tbb; cd tbb</code> and download the latest '*.tar.gz' file from <a href="https://github.com/topkecleon/telegram-bot-bash/releases">https://github.com/topkecleon/telegram-bot-bash/releases</a>. This can be done with the commands:</p>
-<div class="sourceCode" id="cb2"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1"></a><span class="fu">wget</span> -q https://github.com/<span class="va">$(</span><span class="fu">wget</span> -q https://github.com/topkecleon/telegram-bot-bash/releases/latest -O - <span class="kw">|</span> <span class="fu">egrep</span> <span class="st">&#39;/.*/.*/.*tar.gz&#39;</span> -o<span class="va">)</span></span></code></pre></div>
-<p>Extract the '*.tar.gz' file and change to bashbot directory: <code>tar -xzf *.tar.gz; cd telegram-bot-bash</code>, install bashbot: <code>./bashbot.sh init</code> and enter your bot token when asked. All other questions can be answered by hitting the &lt;Return&gt; key.</p>
-<p>That's all, now you can start your bot with <code>./bashbot.sh start</code> and send him messages:</p>
+<h3>Your very first bashbot in a nutshell</h3>
+<p>To install and run bashbot you need access to a Linux/Unix command line with bash, a <a href="https://telegram.org">Telegram client</a> and a mobile phone <a href="https://telegramguide.com/create-a-telegram-account/">with a Telegram account</a>.</p>
+<p>First you need to <a href="doc/1_firstbot.md">create a new Telegram Bot token</a> for your bot and write it down.</p>
+<p>Now open a Linux/Unix terminal with bash, create a new directory, change to it and install telegram-bot-bash:</p>
+<div class="sourceCode" id="cb1"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1"></a><span class="co"># create bot dir</span></span>
+<span id="cb1-2"><a href="#cb1-2"></a><span class="fu">mkdir</span> mybot</span>
+<span id="cb1-3"><a href="#cb1-3"></a><span class="bu">cd</span> mybot</span>
+<span id="cb1-4"><a href="#cb1-4"></a></span>
+<span id="cb1-5"><a href="#cb1-5"></a><span class="co"># download latest release with wget or from https://github.com/topkecleon/telegram-bot-bash/releases/latest</span></span>
+<span id="cb1-6"><a href="#cb1-6"></a><span class="fu">wget</span> <span class="st">&quot;https://github.com/</span><span class="va">$(</span><span class="fu">wget</span> -q <span class="st">&quot;https://github.com/topkecleon/telegram-bot-bash/releases/latest&quot;</span> -O - <span class="kw">|</span> <span class="fu">egrep</span> <span class="st">&#39;/.*/download/.*/.*tar.gz&#39;</span> -o<span class="va">)</span><span class="st">&quot;</span></span>
+<span id="cb1-7"><a href="#cb1-7"></a></span>
+<span id="cb1-8"><a href="#cb1-8"></a><span class="co"># Extract the tar archive and go into bot dir</span></span>
+<span id="cb1-9"><a href="#cb1-9"></a><span class="fu">tar</span> -xzf *.tar.gz</span>
+<span id="cb1-10"><a href="#cb1-10"></a><span class="bu">cd</span> telegram-bot-bash</span>
+<span id="cb1-11"><a href="#cb1-11"></a></span>
+<span id="cb1-12"><a href="#cb1-12"></a><span class="co"># initialize your bot</span></span>
+<span id="cb1-13"><a href="#cb1-13"></a><span class="co"># Enter your bot token when asked, all other questions can be answered by hitting the \&lt;Return\&gt; key.</span></span>
+<span id="cb1-14"><a href="#cb1-14"></a><span class="ex">./bashbot.sh</span> init</span>
+<span id="cb1-15"><a href="#cb1-15"></a></span>
+<span id="cb1-16"><a href="#cb1-16"></a><span class="co"># Now start your bot</span></span>
+<span id="cb1-17"><a href="#cb1-17"></a><span class="ex">./bashbot.sh</span> start</span>
+<span id="cb1-18"><a href="#cb1-18"></a></span>
+<span id="cb1-19"><a href="#cb1-19"></a><span class="ex">Bottoken</span> is valid ...</span>
+<span id="cb1-20"><a href="#cb1-20"></a><span class="ex">Bot</span> Name: yourbotname_bot</span>
+<span id="cb1-21"><a href="#cb1-21"></a><span class="ex">Session</span> Name: yourbotname_bot-startbot</span>
+<span id="cb1-22"><a href="#cb1-22"></a><span class="ex">Bot</span> started successfully.</span></code></pre></div>
+<p>Now open the Telegram App on your mobile phone and start a chat with your bot (<em>your bot's username is shown after 'Bot Name:'</em>):</p>
 <pre><code>/start
 
 You are Botadmin
-*Available commands*:
-*• /start*: _Start bot and get this message_.
-*• /help*: _Get this message_.
-*• /info*: _Get shorter info message about this bot_....
+Available commands:
+  /start: _Start bot and get this message_.
+  /help: _Get this message_.
+  /info: _Get shorter info message about this bot_....
 
 /info
 
 This is bashbot, the Telegram bot written entirely in bash.
 It features background tasks and interactive chats, and can serve as an interface for CLI programs.
 </code></pre>
-<p>For more Information on how to install, customize and use your new bot, read the <a href="#Documentation">Documentation</a></p>
+<p>For more Information on how to install, customize and use your new bot, read the <a href="#Documentation">Documentation</a>.</p>
 <h3>Log files</h3>
-<p>Since version 0.96 bashbot log commands received/send and connection errors. If you start bashbot in debug mode bash stdout, stderr and all send/received telegram message are logged also.</p>
-<p>To enable debug mode start bashbot with debug as third argument: <code>bashbot start debug</code></p>
-<pre><code>├── logs 
-│   ├── BASHBOT.log      # log what your bot is doing ...
-│   ├── ERROR.log        # connection errors from / to telegram API
-│   │
-│   ├── DEBUG.log        # stdout/stderr of you bot (debug mode enabled)
-│   └── MESSAGE.log      # full text of all message send/received (debug mode enabled)
+<p>Bashbot actions are logged to <code>BASHBOT.log</code>. Telegram send/receive errors are logged to <code>ERROR.log</code>. Start bashbot in debug mode to see all messages sent to / received from Telegram, as well as bash command error messages.</p>
+<p>To enable debug mode, start bashbot with debug as third argument: <code>bashbot start debug</code></p>
+<pre><code>|__ logs
+|     |__ BASHBOT.log  # log what your bot is doing ...
+|     |__ ERROR.log    # connection errors from / to Telegram API
+|     |
+|     |__ DEBUG.log    # stdout/stderr of you bot (debug mode enabled)
+|     |__ MESSAGE.log  # full text of all message send/received (debug mode enabled)
 </code></pre>
 <hr />
 <h2>Security Considerations</h2>
 <p>Running a Telegram Bot means it is connected to the public and you never know what's send to your Bot.</p>
-<p>Bash scripts in general are not designed to be bullet proof, so consider this Bot as a proof of concept. Bash programmers often struggle with 'quoting hell' and globbing, see <a href="https://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells">Implications of wrong quoting</a></p>
-<p>Whenever you are processing input from untrusted sources (messages, files, network) you must be as careful as possible, e.g. set IFS appropriate, disable globbing (set -f) and quote everything. In addition delete unused scripts and examples from your Bot, e.g. scripts 'notify', 'calc', 'question', and disable all not used commands.</p>
-<p><strong>Note:</strong> Up to version v0.941 (mai/22/2020) telegram-bot-bash had a remote code execution (RCE) bug, please update if you use an older version! see <a href="https://github.com/topkecleon/telegram-bot-bash/issues/125">Issue #125</a></p>
-<p>One of the most powerful features of unix shells is variable and command substitution using <code>${}</code> and <code>$()</code>, but as they are expanded in double quotes, this can lead to RCE and information disclosing bugs in complex scripts like bashbot. So it's more secure to escape or remove '$' in input from user, files or network.</p>
-<p>A powerful tool to improve your scripts is <code>shellcheck</code>. You can <a href="https://www.shellcheck.net/">use it online</a> or <a href="https://github.com/koalaman/shellcheck#installing">install shellcheck locally</a>. Shellcheck is used extensively in bashbot development to ensure a high code quality, e.g. it's not allowed to push changes without passing all shellcheck tests. In addition bashbot has a <a href="doc/7_develop.md">test suite</a> to check if important functionality is working as expected.</p>
+<p>Bash scripts in general are not designed to be bulletproof, so consider this Bot as a proof of concept. Bash programmers often struggle with 'quoting hell' and globbing, see <a href="https://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells">Implications of wrong quoting</a>.</p>
+<p>Whenever you are processing input from untrusted sources (messages, files, network) you must be as careful as possible (e.g. set IFS appropriately, disable globbing with <code>set -f</code> and quote everything). In addition remove unused scripts and examples from your Bot (e.g. everything in <code>example/</code>) and disable/remove all unused bot commands.</p>
+<p>It's important to escape or remove <code>$</code> and ` in input from user, files or network (<em>as bashbot does</em>). One of the powerful features of Unix shells is variable and command substitution using <code>${var}</code>, <code>$(cmd)</code> and `cmd` can lead to remote code execution (RCE) or remote information disclosure (RID) bugs if unescaped <code>$</code> or ` is included in untrusted input (e.g. <code>$$</code> or <code>$(rm -rf /*)</code>).</p>
+<p>A powerful tool to improve your scripts is <code>shellcheck</code>. You can <a href="https://www.shellcheck.net/">use it online</a> or <a href="https://github.com/koalaman/shellcheck#installing">install shellcheck locally</a>. Shellcheck is used extensively in bashbot development to ensure a high code quality (e.g. it's not allowed to push changes without passing all shellcheck tests). In addition bashbot has a <a href="doc/7_develop.md">test suite</a> to check if important functionality is working as expected.</p>
 <h3>Use printf whenever possible</h3>
-<p>If you're writing a script and it is taking external input (from the user as arguments or file system...), you shouldn't use echo to display it. <a href="https://unix.stackexchange.com/a/6581">Use printf whenever possible</a></p>
-<div class="sourceCode" id="cb5"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1"></a>  <span class="co"># very simple</span></span>
-<span id="cb5-2"><a href="#cb5-2"></a>  <span class="bu">echo</span> <span class="st">&quot;text with variables. PWD=</span><span class="va">$PWD</span><span class="st">&quot;</span></span>
-<span id="cb5-3"><a href="#cb5-3"></a>  <span class="bu">printf</span> <span class="st">&#39;%s\n&#39;</span> <span class="st">&quot;text with variables. PWD=</span><span class="va">$PWD</span><span class="st">&quot;</span></span>
-<span id="cb5-4"><a href="#cb5-4"></a>  <span class="ex">-</span><span class="op">&gt;</span> text with variables. PWD=/home/xxx</span>
-<span id="cb5-5"><a href="#cb5-5"></a></span>
-<span id="cb5-6"><a href="#cb5-6"></a>  <span class="co"># more advanced</span></span>
-<span id="cb5-7"><a href="#cb5-7"></a>  <span class="va">FLOAT=</span><span class="st">&quot;1.2346777892864&quot;</span> <span class="va">INTEGER=</span><span class="st">&quot;12345.123&quot;</span></span>
-<span id="cb5-8"><a href="#cb5-8"></a>  <span class="bu">echo</span> <span class="st">&quot;text with variabeles. float=</span><span class="va">$FLOAT</span><span class="st">, integer=</span><span class="va">$INTEGER</span><span class="st">, PWD=</span><span class="va">$PWD</span><span class="st">&quot;</span></span>
-<span id="cb5-9"><a href="#cb5-9"></a>  <span class="ex">-</span><span class="op">&gt;</span>text with variables. float=1.2346777892864, integer=12345.123, PWD=/home/xxx</span>
-<span id="cb5-10"><a href="#cb5-10"></a></span>
-<span id="cb5-11"><a href="#cb5-11"></a>  <span class="bu">printf</span> <span class="st">&quot;text with variables. float=%.2f, integer=%d, PWD=%s\n&quot;</span> <span class="st">&quot;&quot;</span> <span class="st">&quot;</span><span class="va">$INTEGER</span><span class="st">&quot;</span> <span class="st">&quot;</span><span class="va">$PWD</span><span class="st">&quot;</span></span>
-<span id="cb5-12"><a href="#cb5-12"></a>  <span class="ex">-</span><span class="op">&gt;</span>text with variables. float=1.23, integer=12345, PWD=/home/xxx</span></code></pre></div>
-<h3>Do not use #!/usr/bin/env bash</h3>
-<p><strong>We stay with /bin/bash shebang, because it's more save from security perspective.</strong></p>
-<p>Use of a fixed path to the system provided bash makes it harder for attackers or users to place alternative versions of bash and avoids using a possibly broken, mangled or compromised bash executable.</p>
-<p>If you are a BSD / MacOS user or must to use an other bash location, see <a href="doc/0_install.md">Install Bashbot</a></p>
+<p>If you're writing a script that accepts external input (e.g. from the user as arguments or the file system), you shouldn't use echo to display it. <a href="https://unix.stackexchange.com/a/6581">Use printf whenever possible</a>.</p>
 <h3>Run your Bot as a restricted user</h3>
-<p><strong>I recommend to run your bot as a user, with almost no access rights.</strong> All files your Bot have write access to are in danger to be overwritten/deleted if your bot is hacked. For the same reason every file your Bot can read is in danger to be disclosed. Restrict your Bots access rights to the absolute minimum.</p>
-<p><strong>Never run your Bot as root, this is the most dangerous you can do!</strong> Usually the user 'nobody' has almost no rights on unix/linux systems. See <a href="doc/4_expert.md">Expert use</a> on how to run your Bot as an other user.</p>
+<p><strong>I recommend running your bot as a user with almost no access rights.</strong> All files your Bot has write access to are in danger of being overwritten/deleted if your bot is hacked. For the same reason every file your Bot can read is in danger of being disclosed. Restrict your Bots access rights to the absolute minimum.</p>
+<p><strong>Never run your Bot as root, this is the most dangerous you can do!</strong> Usually the user 'nobody' has almost no rights on Linux/Unix systems. See <a href="doc/4_expert.md">Expert use</a> on how to run your Bot as an other user.</p>
 <h3>Secure your Bot installation</h3>
-<p><strong>Your Bot configuration must no be readable from other users.</strong> Everyone who can read your Bots token is able to act as your Bot and has access to all chats the Bot is in!</p>
-<p>Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in <code>config.jssh</code> must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to <code>count.jssh</code> and <code>data-bot-bash</code> only, all other files must be write protected.</p>
+<p><strong>Your Bot configuration must not be readable by other users.</strong> Everyone who can read your Bots token is able to act as your Bot and has access to all chats the Bot is in!</p>
+<p>Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in <code>config.jssh</code> must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to <code>count.jssh</code>, <code>data-bot-bash/</code> and <code>logs/</code> only, all other files must be write protected.</p>
 <p>To set access rights for your bashbot installation to a reasonable default run <code>sudo ./bashbot.sh init</code> after every update or change to your installation directory.</p>
+<p><em>Note</em>: Keep old log files in a safe place or even better delete them, they are GDPR relevant and <a href="https://github.com/topkecleon/telegram-bot-bash/issues/174">may contain information</a> you don't want to be public.</p>
 <h2>FAQ</h2>
 <h3>Is this Bot insecure?</h3>
-<p>Bashbot is not more (in)secure as any Bot written in an other language, we have done our best to make it as secure as possible. But YOU are responsible for the bot commands you wrote and you should know about the risks ...</p>
+<p>Bashbot is not more (in)secure than a Bot written in another language. We have done our best to make it as secure as possible. But YOU are responsible for the bot commands you wrote and you should know about the risks ...</p>
 <p><strong>Note:</strong> Up to version 0.941 (mai/22/2020) telegram-bot-bash had a remote code execution bug, please update if you use an older version!</p>
 <h3>Why Bash and not the much better xyz?</h3>
-<p>Well, that's a damn good question ... may be because I'm an unix admin from stone age. Nevertheless there are more reasons from my side:</p>
+<p>Well, that's a damn good question... maybe because I'm a Unix admin from the stone age. Nevertheless there are more reasons from my side:</p>
 <ul>
-<li>bashbot will run everywhere where bash and (gnu) sed is available, from embedded linux to mainframe</li>
-<li>easy to integrate with other shell script, e.g. for sending system message / health status</li>
+<li>bashbot will run wherever bash and (gnu) sed is available, from embedded Linux to mainframe</li>
+<li>easy to integrate with other shell scripts, e.g. for sending system message / health status</li>
 <li>no need to install or learn a new programming language, library or framework</li>
 <li>no database, not event driven, not object oriented ...</li>
 </ul>
 <h3>Can I have the single bashbot.sh file back?</h3>
-<p>At the beginning bashbot was simply the file <code>bashbot.sh</code> you can copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.</p>
-<p>Hey no Problem, if you are finished with your cool bot run <code>dev/make-standalone.sh</code> to create a stripped down Version of your bot containing only 'bashbot.sh' and 'commands.sh'! For more information see <a href="doc/7_develop.md">Create a stripped down Version of your Bot</a></p>
+<p>At the beginning bashbot was simply the file <code>bashbot.sh</code> that you could copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.</p>
+<p>Hey no problem, if you are finished with your cool bot, run <code>dev/make-standalone.sh</code> to create a stripped down version of your bot containing only 'bashbot.sh' and 'commands.sh'! For more information see <a href="doc/7_develop.md">Create a stripped down version of your Bot</a>.</p>
 <h3>Can I send messages from CLI and scripts?</h3>
-<p>Of course, you can send messages from CLI and scripts, simply install bashbot as <a href="#Your-really-first-bashbot-in-a-nutshell">described here</a>, send the message '/start' to set yourself as botadmin and stop the bot with <code>./bashbot.sh stop</code>.</p>
-<p>Run the following commands in your bash shell or script while you are in the installation directory:</p>
-<div class="sourceCode" id="cb6"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1"></a><span class="co"># prepare bash / script to send commands</span></span>
-<span id="cb6-2"><a href="#cb6-2"></a><span class="bu">export</span> <span class="va">BASHBOT_HOME=</span><span class="st">&quot;</span><span class="va">$(</span><span class="bu">pwd</span><span class="va">)</span><span class="st">&quot;</span></span>
-<span id="cb6-3"><a href="#cb6-3"></a><span class="bu">source</span> ./bashbot.sh source</span>
-<span id="cb6-4"><a href="#cb6-4"></a></span>
-<span id="cb6-5"><a href="#cb6-5"></a><span class="co"># send me a test message</span></span>
-<span id="cb6-6"><a href="#cb6-6"></a><span class="ex">send_message</span> <span class="st">&quot;</span><span class="va">$(</span><span class="ex">getConfigKey</span> <span class="st">&quot;botadmin&quot;</span><span class="va">)</span><span class="st">&quot;</span> <span class="st">&quot;test&quot;</span></span>
-<span id="cb6-7"><a href="#cb6-7"></a></span>
-<span id="cb6-8"><a href="#cb6-8"></a><span class="co"># send me output of a system command</span></span>
-<span id="cb6-9"><a href="#cb6-9"></a><span class="ex">send_message</span> <span class="st">&quot;</span><span class="va">$(</span><span class="ex">getConfigKey</span> <span class="st">&quot;botadmin&quot;</span><span class="va">)</span><span class="st">&quot;</span> <span class="st">&quot;</span><span class="va">$(</span><span class="fu">df</span> -h<span class="va">)</span><span class="st">&quot;</span></span></code></pre></div>
-<p>For more information see <a href="doc/8_custom.md">Expert Use</a></p>
+<p>Of course you can send messages from command line and scripts! Simply install bashbot as <a href="#Your-really-first-bashbot-in-a-nutshell">described here</a>, send the message '/start' to set yourself as botadmin and then stop the bot with <code>./bashbot.sh stop</code>.</p>
+<p>Bashbot provides some ready to use scripts for sending messages from command line in <code>bin/</code> dir, e.g. <code>send_message.sh</code>.</p>
+<div class="sourceCode" id="cb4"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1"></a><span class="ex">bin/send_message.sh</span> BOTADMIN <span class="st">&quot;This is my first message send from CLI&quot;</span></span>
+<span id="cb4-2"><a href="#cb4-2"></a></span>
+<span id="cb4-3"><a href="#cb4-3"></a><span class="ex">bin/send_message.sh</span> --help</span></code></pre></div>
+<p>You can also source bashbot for use in your scripts, for more information see <a href="doc/4_expert.md">Expert Use</a>.</p>
 <h3>Blocked by telegram?</h3>
-<p>This may happen if to many or wrong requests are sent to api.telegram.org, e.g. using a invalid token or not existing API calls. If the block stay for longer time you can ask telegram service to unblock your IP-Adress.</p>
+<p>This may happen if too many or wrong requests are sent to api.telegram.org, e.g. using a invalid token or invalid API calls. If the block stay for longer time you can ask telegram service to unblock your IP-Address.</p>
 <p>You can check with curl or wget if you are blocked by Telegram:</p>
-<div class="sourceCode" id="cb7"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1"></a><span class="ex">curl</span> -m 10  https://api.telegram.org/bot</span>
-<span id="cb7-2"><a href="#cb7-2"></a><span class="co">#curl: (28) Connection timed out after 10001 milliseconds</span></span>
-<span id="cb7-3"><a href="#cb7-3"></a></span>
-<span id="cb7-4"><a href="#cb7-4"></a><span class="fu">wget</span> -t 1 -T 10 https://api.telegram.org/bot</span>
-<span id="cb7-5"><a href="#cb7-5"></a><span class="co">#Connecting to api.telegram.org (api.telegram.org)|46.38.243.234|:443... failed: Connection timed out.</span></span></code></pre></div>
-<p>Since Version 0.96 bashbot offers the option to recover from broken connections (aka blocked). Therefore you can provide a function named <code>bashbotBlockRecover()</code> in <code>mycommands.sh</code>. If the function exists it is called every time when a broken connection is detected.</p>
-<p>Possible actions are: Check if network is working, change IP-Adress or simply wait some time.</p>
-<p>If everything seems OK return 0 for retry or any non 0 value to give up.</p>
-<div class="sourceCode" id="cb8"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb8-1"><a href="#cb8-1"></a><span class="co"># called when bashbot sedn command failed because we can not connect to telegram</span></span>
-<span id="cb8-2"><a href="#cb8-2"></a><span class="co"># return 0 to retry, return non 0 to give up</span></span>
-<span id="cb8-3"><a href="#cb8-3"></a><span class="fu">bashbotBlockRecover()</span> <span class="kw">{</span></span>
-<span id="cb8-4"><a href="#cb8-4"></a>    <span class="co"># place your commands to unblock here, e.g. change IP-Adess or simply wait</span></span>
-<span id="cb8-5"><a href="#cb8-5"></a>    <span class="fu">sleep</span> 60 <span class="kw">&amp;&amp;</span> <span class="bu">return</span> 0 <span class="co"># may be temporary</span></span>
-<span id="cb8-6"><a href="#cb8-6"></a>    <span class="bu">return</span> 1 </span>
-<span id="cb8-7"><a href="#cb8-7"></a>    <span class="kw">}</span></span>
-<span id="cb8-8"><a href="#cb8-8"></a></span></code></pre></div>
+<div class="sourceCode" id="cb5"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1"></a><span class="ex">curl</span> -m 10  https://api.telegram.org/bot</span>
+<span id="cb5-2"><a href="#cb5-2"></a><span class="co">#curl: (28) Connection timed out after 10001 milliseconds</span></span>
+<span id="cb5-3"><a href="#cb5-3"></a></span>
+<span id="cb5-4"><a href="#cb5-4"></a><span class="fu">wget</span> -t 1 -T 10 https://api.telegram.org/bot</span>
+<span id="cb5-5"><a href="#cb5-5"></a><span class="co">#Connecting to api.telegram.org (api.telegram.org)|46.38.243.234|:443... failed: Connection timed out.</span></span>
+<span id="cb5-6"><a href="#cb5-6"></a></span>
+<span id="cb5-7"><a href="#cb5-7"></a><span class="ex">nc</span> -w 2 api.telegram.org 443 <span class="kw">||</span> <span class="bu">echo</span> <span class="st">&quot;your IP seems blocked by telegram&quot;</span></span>
+<span id="cb5-8"><a href="#cb5-8"></a><span class="co">#your IP seems blocked by telegram</span></span></code></pre></div>
+<p>Bashbot offers the option to recover from broken connections (blocked). Therefore you can provide a function named <code>bashbotBlockRecover()</code> in <code>mycommands.sh</code>, the function is called every time when a broken connection is detected.</p>
+<p>Possible actions are: Check if network is working, change IP-Address or simply wait some time. See <code>mycommnds.sh.dist</code> for an example.</p>
+<hr />
 <p>@Gnadelwartz</p>
 <h2>That's it all guys!</h2>
 <p>If you feel that there's something missing or if you found a bug, feel free to submit a pull request!</p>
-<h4>$$VERSION$$ v1.1-0-ge835bbc</h4>
+<h4>$$VERSION$$ v1.52-0-g1a83202</h4>
 </body>
 </html>

README.md

@@ -3,24 +3,32 @@ Bashbot - A Telegram bot written in bash.
 </h2>
 Written by Drew (@topkecleon) and Kay M (@gnadelwartz).
 
-Contributions by Daniil Gentili (@danogentili), JuanPotato, BigNerd95, TiagoDanin, and iicc1.
+Contributions by Daniil Gentili (@danog), JuanPotato, BigNerd95, TiagoDanin, iicc1 and dcoomber.
 
 Released to the public domain wherever applicable.
 Elsewhere, consider it released under the [WTFPLv2](http://www.wtfpl.net/txt/copying/).
 
 Linted by [#ShellCheck](https://github.com/koalaman/shellcheck)
 
+---
+**Bashbot** is created by old-fashioned shell hackers to show that it's possible to write a bot in Bash.
+
+It is designed for **simple** use cases and easily integrates with Linux tasks.
+For bots that serve a lot of users or a heavy workload, I strongly recommend using a dedicated bot framework.
+
+---
+
 ## Prerequisites
-Uses [JSON.sh](http://github.com/dominictarr/JSON.sh) and the magic of sed.
+Uses [JSON.sh](http://github.com/dominictarr/JSON.sh)/[JSON.awk](https://github.com/step-/JSON.awk) and the magic of sed.
 
-Even bashbot is written in bash, it depends on commands typically available in a Unix/Linux Environment.
-More concrete on the common commands provided by recent versions of [coreutils](https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands), [busybox](https://en.wikipedia.org/wiki/BusyBox#Commands) or [toybox](https://landley.net/toybox/help.html), see [Developer Notes](doc/7_develop.md#common-commands)
+Bashbot is written in bash. It depends on commands typically available in a Linux/Unix Environment.
+For more information on commands provided by recent versions of [coreutils](https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands), [busybox](https://en.wikipedia.org/wiki/BusyBox#Commands) or [toybox](https://landley.net/toybox/help.html), see [Developer Notes](doc/7_develop.md#common-commands).
 
-**Note for MacOS and BSD Users:** As bashbot heavily uses modern bash and (gnu) grep/sed features, bashbot will not run without installing additional software, see [Install Bashbot](doc/0_install.md)
+**Note for MacOS and BSD Users:** Bashbot will not run without installing additional software as it uses modern bash and (gnu) grep/sed features. See [Install Bashbot](doc/0_install.md).
 
-**Note for embedded systems:** busybox or toybox ONLY is not sufficient, you need a to install a "real" bash, see also [Install Bashbot](doc/0_install.md)  
+**Note for embedded systems:** You need to install a "real" bash as the vanilla installation of busybox or toybox is not sufficient. See [Install Bashbot](doc/0_install.md).
 
-Bashbot [Documentation](https://github.com/topkecleon/telegram-bot-bash) and [Downloads](https://github.com/topkecleon/telegram-bot-bash/releases) are available on www.github.com
+Bashbot [Documentation](https://github.com/topkecleon/telegram-bot-bash) and [Downloads](https://github.com/topkecleon/telegram-bot-bash/releases) are available on [www.github.com](https://www.github.com).
 
 ## Documentation
 * [Introduction to Telegram Bots](https://core.telegram.org/bots)
@@ -63,65 +71,74 @@ Bashbot [Documentation](https://github.com/topkecleon/telegram-bot-bash) and [Do
     * Modules, addons, events
     * Setup your environment
     * Bashbot test suite
-* [Examples Dir](examples/README.md)
+* [Examples Directory](examples)
+* [Webhook Example](examples/webhook)
 
-### Your really first bashbot in a nutshell
+### Your very first bashbot in a nutshell
 
-To install and run bashbot you need access to a linux/unix command line. If you don't know how to get access to a linux/unix command line you should stop reading here :-(
+To install and run bashbot you need access to a Linux/Unix command line with bash, a [Telegram client](https://telegram.org) and a mobile phone [with a Telegram account](https://telegramguide.com/create-a-telegram-account/).
 
-In addition you need a [Telegram client](https://telegram.org) and a mobile phone to [register an account](https://telegramguide.com/create-a-telegram-account/).
-If you don't want to register for Telegram you should stop reading here ;-)
+First you need to [create a new Telegram Bot token](doc/1_firstbot.md) for your bot and write it down.
 
-After you're registered to Telegram send a message to [@botfather](https://telegram.me/botfather),
-[create a new Telegram Bot token](doc/1_firstbot.md) and write it down. You need the token to install the bot.
+Now open a Linux/Unix terminal with bash, create a new directory, change to it and install telegram-bot-bash:
 
-Now open a terminal and check if bash is installed:
- ```
-which bash && echo "bash seems available..."
-```
-
-
-Create a new directory, change to it:  ```mkdir tbb; cd tbb``` and download the latest '*.tar.gz' file from
-[https://github.com/topkecleon/telegram-bot-bash/releases](https://github.com/topkecleon/telegram-bot-bash/releases). This can be done with the commands:
 ```bash
-wget -q https://github.com/$(wget -q https://github.com/topkecleon/telegram-bot-bash/releases/latest -O - | egrep '/.*/.*/.*tar.gz' -o)
+# create bot dir
+mkdir mybot
+cd mybot
+
+# download latest release with wget or from https://github.com/topkecleon/telegram-bot-bash/releases/latest
+wget "https://github.com/$(wget -q "https://github.com/topkecleon/telegram-bot-bash/releases/latest" -O - | egrep '/.*/download/.*/.*tar.gz' -o)"
+
+# Extract the tar archive and go into bot dir
+tar -xzf *.tar.gz
+cd telegram-bot-bash
+
+# initialize your bot
+# Enter your bot token when asked, all other questions can be answered by hitting the \<Return\> key.
+./bashbot.sh init
+
+# Now start your bot
+./bashbot.sh start
+
+Bottoken is valid ...
+Bot Name: yourbotname_bot
+Session Name: yourbotname_bot-startbot
+Bot started successfully.
 ```
 
-Extract the '*.tar.gz' file and change to bashbot directory: ```tar -xzf *.tar.gz; cd telegram-bot-bash```,
-install bashbot: ```./bashbot.sh init``` and enter your bot token when asked. All other questions can be answered
-by hitting the \<Return\> key.
+Now open the Telegram App on your mobile phone and start a chat with your bot (_your bot's username is shown after 'Bot Name:'_):
 
-That's all, now you can start your bot with ```./bashbot.sh start``` and send him messages:
 ```
 /start
 
 You are Botadmin
-*Available commands*:
-*• /start*: _Start bot and get this message_.
-*• /help*: _Get this message_.
-*• /info*: _Get shorter info message about this bot_....
+Available commands:
+  /start: _Start bot and get this message_.
+  /help: _Get this message_.
+  /info: _Get shorter info message about this bot_....
 
 /info
 
 This is bashbot, the Telegram bot written entirely in bash.
 It features background tasks and interactive chats, and can serve as an interface for CLI programs.
 ```
-For more Information on how to install, customize and use your new bot, read the [Documentation](#Documentation)
+For more Information on how to install, customize and use your new bot, read the [Documentation](#Documentation).
 
 ### Log files
 
-Since version 0.96 bashbot log commands received/send and connection errors. If you start bashbot in debug mode
-bash stdout, stderr and all send/received telegram message are logged also.
+Bashbot actions are logged to `BASHBOT.log`. Telegram send/receive errors are logged to `ERROR.log`.
+Start bashbot in debug mode to see all messages sent to / received from Telegram, as well as bash command error messages.
 
-To enable debug mode start bashbot with debug as third argument: `bashbot start debug`
+To enable debug mode, start bashbot with debug as third argument: `bashbot start debug`
 
 ```
-├── logs 
-│   ├── BASHBOT.log      # log what your bot is doing ...
-│   ├── ERROR.log        # connection errors from / to telegram API
-│   │
-│   ├── DEBUG.log        # stdout/stderr of you bot (debug mode enabled)
-│   └── MESSAGE.log      # full text of all message send/received (debug mode enabled)
+|__ logs
+|     |__ BASHBOT.log  # log what your bot is doing ...
+|     |__ ERROR.log    # connection errors from / to Telegram API
+|     |
+|     |__ DEBUG.log    # stdout/stderr of you bot (debug mode enabled)
+|     |__ MESSAGE.log  # full text of all message send/received (debug mode enabled)
 ```
 
 ----
@@ -129,107 +146,83 @@ To enable debug mode start bashbot with debug as third argument: `bashbot start
 ## Security Considerations
 Running a Telegram Bot means it is connected to the public and you never know what's send to your Bot.
 
-Bash scripts in general are not designed to be bullet proof, so consider this Bot as a proof of concept. Bash programmers often struggle with 'quoting hell' and globbing, see [Implications of wrong quoting](https://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells)
+Bash scripts in general are not designed to be bulletproof, so consider this Bot as a proof of concept.
+Bash programmers often struggle with 'quoting hell' and globbing,
+see [Implications of wrong quoting](https://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells).
 
-Whenever you are processing input from untrusted sources (messages, files, network) you must be as careful as possible, e.g. set IFS appropriate, disable globbing (set -f) and quote everything. In addition delete unused scripts and examples from your Bot, e.g. scripts 'notify', 'calc', 'question', and disable all not used commands.
+Whenever you are processing input from untrusted sources (messages, files, network) you must be as careful as possible 
+(e.g. set IFS appropriately, disable globbing with `set -f` and quote everything). In addition remove unused scripts and examples
+from your Bot (e.g. everything in `example/`) and disable/remove all unused bot commands.
 
-**Note:** Up to version v0.941 (mai/22/2020) telegram-bot-bash had a remote code execution (RCE) bug, please update if you use an older version!
-see [Issue #125](https://github.com/topkecleon/telegram-bot-bash/issues/125)
+It's important to escape or remove `$` and \` in input from user, files or network (_as bashbot does_).
+One of the powerful features of Unix shells is variable and command substitution using `${var}`, `$(cmd)` and \`cmd\` can lead to remote
+code execution (RCE) or remote information disclosure (RID) bugs if unescaped `$` or \` is included in untrusted input (e.g. `$$` or `$(rm -rf /*)`).
 
-One of the most powerful features of unix shells is variable and command substitution using ```${}``` and ```$()```,
-but as they are expanded in double quotes, this can lead to RCE and information disclosing bugs in complex scripts like bashbot.
-So it's more secure to escape or remove '$' in input from user, files or network.
-
-A powerful tool to improve your scripts is ```shellcheck```. You can [use it online](https://www.shellcheck.net/) or [install shellcheck locally](https://github.com/koalaman/shellcheck#installing). Shellcheck is used extensively in bashbot development to ensure a high code quality, e.g. it's not allowed to push changes without passing all shellcheck tests.
+A powerful tool to improve your scripts is `shellcheck`. You can [use it online](https://www.shellcheck.net/) or
+[install shellcheck locally](https://github.com/koalaman/shellcheck#installing). Shellcheck is used extensively in bashbot development
+to ensure a high code quality (e.g. it's not allowed to push changes without passing all shellcheck tests).
 In addition bashbot has a [test suite](doc/7_develop.md) to check if important functionality is working as expected.
 
 ### Use printf whenever possible
 
-If you're writing a script and it is taking external input (from the user as arguments or file system...),
-you shouldn't use echo to display it. [Use printf whenever possible](https://unix.stackexchange.com/a/6581)
-
-```bash
-  # very simple
-  echo "text with variables. PWD=$PWD"
-  printf '%s\n' "text with variables. PWD=$PWD"
-  -> text with variables. PWD=/home/xxx
-
-  # more advanced
-  FLOAT="1.2346777892864" INTEGER="12345.123"
-  echo "text with variabeles. float=$FLOAT, integer=$INTEGER, PWD=$PWD"
-  ->text with variables. float=1.2346777892864, integer=12345.123, PWD=/home/xxx
-
-  printf "text with variables. float=%.2f, integer=%d, PWD=%s\n" "" "$INTEGER" "$PWD"
-  ->text with variables. float=1.23, integer=12345, PWD=/home/xxx
-```
-
-### Do not use #!/usr/bin/env bash
-
-**We stay with /bin/bash shebang, because it's more save from security perspective.**
-
-Use of a fixed path to the system provided bash makes it harder for attackers or users to place alternative versions of bash
-and avoids using a possibly broken, mangled or compromised bash executable. 
-
-If you are a BSD /  MacOS user or must to use an other bash location, see [Install Bashbot](doc/0_install.md)
+If you're writing a script that accepts external input (e.g. from the user as arguments or the file system),
+you shouldn't use echo to display it. [Use printf whenever possible](https://unix.stackexchange.com/a/6581).
 
 ### Run your Bot as a restricted user
-**I recommend to run your bot as a user, with almost no access rights.** 
-All files your Bot have write access to are in danger to be overwritten/deleted if your bot is hacked.
-For the same reason every file your Bot can read is in danger to be disclosed. Restrict your Bots access rights to the absolute minimum.
+**I recommend running your bot as a user with almost no access rights.** 
+All files your Bot has write access to are in danger of being overwritten/deleted if your bot is hacked.
+For the same reason every file your Bot can read is in danger of being disclosed. Restrict your Bots access rights to the absolute minimum.
 
-**Never run your Bot as root, this is the most dangerous you can do!** Usually the user 'nobody' has almost no rights on unix/linux systems. See [Expert use](doc/4_expert.md) on how to run your Bot as an other user.
+**Never run your Bot as root, this is the most dangerous you can do!** Usually the user 'nobody' has almost no rights on Linux/Unix systems. See [Expert use](doc/4_expert.md) on how to run your Bot as an other user.
 
 ### Secure your Bot installation
-**Your Bot configuration must no be readable from other users.** Everyone who can read your Bots token is able to act as your Bot and has access to all chats the Bot is in!
+**Your Bot configuration must not be readable by other users.** Everyone who can read your Bots token is able to act as your Bot and has access to all chats the Bot is in!
 
-Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in ```config.jssh``` must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to ```count.jssh``` and  ```data-bot-bash``` only, all other files must be write protected.
+Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in `config.jssh` must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to `count.jssh`, `data-bot-bash/` and `logs/` only, all other files must be write protected.
 
-To set access rights for your bashbot installation to a reasonable default run ```sudo ./bashbot.sh init``` after every update or change to your installation directory.
+To set access rights for your bashbot installation to a reasonable default run `sudo ./bashbot.sh init` after every update or change to your installation directory.
+
+*Note*: Keep old log files in a safe place or even better delete them, they are GDPR relevant and [may contain information](https://github.com/topkecleon/telegram-bot-bash/issues/174) you don't want to be public.
 
 ## FAQ
 
 ### Is this Bot insecure?
-Bashbot is not more (in)secure as any Bot written in an other language, we have done our best to make it as secure as possible. But YOU are responsible for the bot commands you wrote and you should know about the risks ...
+Bashbot is not more (in)secure than a Bot written in another language. We have done our best to make it as secure as possible. But YOU are responsible for the bot commands you wrote and you should know about the risks ...
 
 **Note:** Up to version 0.941 (mai/22/2020) telegram-bot-bash had a remote code execution bug, please update if you use an older version!
 
 ### Why Bash and not the much better xyz?
-Well, that's a damn good question ... may be because I'm an unix admin from stone age. Nevertheless there are more reasons from my side:
+Well, that's a damn good question... maybe because I'm a Unix admin from the stone age. Nevertheless there are more reasons from my side:
 
-- bashbot will run everywhere where bash and (gnu) sed is available, from embedded linux to mainframe
-- easy to integrate with other shell script, e.g. for sending system message / health status
+- bashbot will run wherever bash and (gnu) sed is available, from embedded Linux to mainframe
+- easy to integrate with other shell scripts, e.g. for sending system message / health status
 - no need to install or learn a new programming language, library or framework
 - no database, not event driven, not object oriented ...
 
 ### Can I have the single bashbot.sh file back?
-At the beginning bashbot was simply the file ```bashbot.sh``` you can copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.
+At the beginning bashbot was simply the file `bashbot.sh` that you could copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.
 
-Hey no Problem, if you are finished with your cool bot run ```dev/make-standalone.sh``` to create a stripped down Version of your bot containing only
-'bashbot.sh' and 'commands.sh'! For more information see [Create a stripped down Version of your Bot](doc/7_develop.md)
+Hey no problem, if you are finished with your cool bot, run `dev/make-standalone.sh` to create a stripped down version of your bot containing only
+'bashbot.sh' and 'commands.sh'! For more information see [Create a stripped down version of your Bot](doc/7_develop.md).
 
 ### Can I send messages from CLI and scripts?
-Of course, you can send messages from CLI and scripts, simply install bashbot as [described here](#Your-really-first-bashbot-in-a-nutshell),
-send the message '/start' to set yourself as botadmin and stop the bot with ```./bashbot.sh stop```.
+Of course you can send messages from command line and scripts! Simply install bashbot as [described here](#Your-really-first-bashbot-in-a-nutshell),
+send the message '/start' to set yourself as botadmin and then stop the bot with `./bashbot.sh stop`.
 
-Run the following commands in your bash shell or script while you are in the installation directory:
+Bashbot provides some ready to use scripts for sending messages from command line in `bin/` dir, e.g. `send_message.sh`.
 
 ```bash
-# prepare bash / script to send commands
-export BASHBOT_HOME="$(pwd)"
-source ./bashbot.sh source
+bin/send_message.sh BOTADMIN "This is my first message send from CLI"
 
-# send me a test message
-send_message "$(getConfigKey "botadmin")" "test"
-
-# send me output of a system command
-send_message "$(getConfigKey "botadmin")" "$(df -h)"
+bin/send_message.sh --help
 ```
-For more information see [Expert Use](doc/8_custom.md)
+
+You can also source bashbot for use in your scripts, for more information see [Expert Use](doc/4_expert.md).
 
 
 ### Blocked by telegram?
-This may happen if to many or wrong requests are sent to api.telegram.org, e.g. using a invalid token or not existing API calls.
-If the block stay for longer time you can ask telegram service to unblock your IP-Adress. 
+This may happen if too many or wrong requests are sent to api.telegram.org, e.g. using a invalid token or invalid API calls.
+If the block stay for longer time you can ask telegram service to unblock your IP-Address. 
 
 You can check with curl or wget if you are blocked by Telegram:
 ```bash
@@ -238,26 +231,18 @@ curl -m 10  https://api.telegram.org/bot
 
 wget -t 1 -T 10 https://api.telegram.org/bot
 #Connecting to api.telegram.org (api.telegram.org)|46.38.243.234|:443... failed: Connection timed out.
+
+nc -w 2 api.telegram.org 443 || echo "your IP seems blocked by telegram"
+#your IP seems blocked by telegram
 ```
 
-Since Version 0.96 bashbot offers the option to recover from broken connections (aka blocked). Therefore you can provide a function
-named `bashbotBlockRecover()` in `mycommands.sh`. If the function exists it is called every time when a broken connection is detected.
+Bashbot offers the option to recover from broken connections (blocked). Therefore you can provide a function
+named `bashbotBlockRecover()` in `mycommands.sh`, the function is called every time when a broken connection is detected.
 
-Possible actions are: Check if network is working, change IP-Adress or simply wait some time.
+Possible actions are: Check if network is working, change IP-Address or simply wait some time.
+See `mycommnds.sh.dist` for an example.
 
-If everything seems OK return 0 for retry or any non 0 value to give up.
-
-```bash
-# called when bashbot sedn command failed because we can not connect to telegram
-# return 0 to retry, return non 0 to give up
-bashbotBlockRecover() {
-	# place your commands to unblock here, e.g. change IP-Adess or simply wait
-	sleep 60 && return 0 # may be temporary
-	return 1 
-    }
-
-```
- 
+--- 
 
 @Gnadelwartz
 
@@ -265,4 +250,4 @@ bashbotBlockRecover() {
 
 If you feel that there's something missing or if you found a bug, feel free to submit a pull request!
 
-#### $$VERSION$$ v1.1-0-ge835bbc
+#### $$VERSION$$ v1.52-1-g0dae2db

README.txt

@@ -1,358 +1,323 @@
-<h2><img align="middle" 
-src="https://raw.githubusercontent.com/odb/official-bash-logo/master/assets/Logo
-s/Icons/PNG/64x64.png" >
-Bashbot - A Telegram bot written in bash.
-</h2>
+
+                      ..
+                     ****                     
+                ****oooooo*****             
+           *****ooooooooooooo*****        
+       *****oooooooooooooooooooooo****  
+   ****oooooooooooooooooooooooooooooooo**
+ *.*oooooooooooooooooooooooooooooooooooo**
+ *.ooooooooooooooooooooooooooooooooo**....
+ *.oooooooooooooooooooooooooooo**.........
+ *.oooooooooooooooooooooooo**.............
+ *.ooooooooooooooooooo**..................     ____               _      _             _   
+ *.ooooooooooooooooo*.......,.............    |  _ \             | |    | |           | |  
+ *.ooooooooooooooooo*.....,***,...........    | |_) |  __ _  ___ | |__  | |__    ___  | |_ 
+ *.ooooooooooooooooo*....o*...............    |  _ <  / _` |/ __|| '_ \ | '_ \  / _ \ | __|
+ *.ooooooooooooooooo*....*o***,...........    | |_) || (_| |\__ \| | | || |_) || (_) || |_ 
+ *.*oooooooooooooooo*........o*.....oo....    |____/  \__,_||___/|_| |_||_.__/  \___/  \__|
+   ****ooooooooooooo*....`***....oo.....*
+       *****oooooooo*......*..oo.....**  
+           ******ooo*.............*       
+                ***o*........**             
+                    **...**                    
+
+
+
+
+
+Bashbot README
+
+
+ Bashbot - A Telegram bot written in bash.
+
 Written by Drew (@topkecleon) and Kay M (@gnadelwartz).
+Contributions by Daniil Gentili (@danog), JuanPotato, BigNerd95, TiagoDanin, iicc1 and
+dcoomber.
+Released to the public domain wherever applicable. Elsewhere, consider it released under
+the WTFPLv2 [http://www.wtfpl.net/txt/copying/].
+Linted by #ShellCheck
 
-Contributions by Daniil Gentili (@danogentili), JuanPotato, BigNerd95, 
-TiagoDanin, and iicc1.
+Prerequisites
 
-Released to the public domain wherever applicable.
-Elsewhere, consider it released under the 
-[WTFPLv2](http://www.wtfpl.net/txt/copying/).
+Uses JSON.sh [http://github.com/dominictarr/JSON.sh]/JSON.awk [https://github.com/step-/
+JSON.awk] and the magic of sed.
+Bashbot is written in bash. It depends on commands typically available in a Linux/Unix
+Environment. For more information on commands provided by recent versions of coreutils
+[https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands], busybox [https://
+en.wikipedia.org/wiki/BusyBox#Commands] or toybox [https://landley.net/toybox/help.html],
+see Developer Notes [doc/7_develop.md#common-commands].
+Note for MacOS and BSD Users: Bashbot will not run without installing additional software
+as it uses modern bash and (gnu) grep/sed features. See Install Bashbot [doc/
+0_install.md].
+Note for embedded systems: You need to install a "real" bash as the vanilla installation
+of busybox or toybox is not sufficient. See Install Bashbot [doc/0_install.md].
+Bashbot Documentation [https://github.com/topkecleon/telegram-bot-bash] and Downloads
+[https://github.com/topkecleon/telegram-bot-bash/releases] are available on www.github.com
+[https://www.github.com].
 
-Linted by [#ShellCheck](https://github.com/koalaman/shellcheck)
-
-## Prerequisites
-Uses [JSON.sh](http://github.com/dominictarr/JSON.sh) and the magic of sed.
-
-Even bashbot is written in bash, it depends on commands typically available in 
-a Unix/Linux Environment.
-More concrete on the common commands provided by recent versions of 
-[coreutils](https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands), 
-[busybox](https://en.wikipedia.org/wiki/BusyBox#Commands) or 
-[toybox](https://landley.net/toybox/help.html), see [Developer 
-Notes](doc/7_develop.md#common-commands)
-
-**Note for MacOS and BSD Users:** As bashbot heavily uses modern bash and (gnu) 
-grep/sed features, bashbot will not run without installing additional software, 
-see [Install Bashbot](doc/0_install.md)
-
-**Note for embedded systems:** busybox or toybox ONLY is not sufficient, you 
-need a to install a "real" bash, see also [Install Bashbot](doc/0_install.md)  
-
-Bashbot [Documentation](https://github.com/topkecleon/telegram-bot-bash) and 
-[Downloads](https://github.com/topkecleon/telegram-bot-bash/releases) are 
-available on www.github.com
-
-## Documentation
-* [Introduction to Telegram Bots](https://core.telegram.org/bots)
-* [Install Bashbot](doc/0_install.md)
-    * Install release
-    * Install from github
-    * Update Bashbot
-    * Notes on Updates
-* [Get Bottoken from Botfather](doc/1_firstbot.md)
-* [Getting Started](doc/2_usage.md)
-    * Managing your Bot
-    * Receive data
-    * Send messages
-    * Send files, locations, keyboards
-* [Advanced Features](doc/3_advanced.md)
-    * Access Control
-    * Interactive Chats
-    * Background Jobs
-    * Inline queries
-    * Send message errors
-* [Expert Use](doc/4_expert.md)
-    * Handling UTF-8 character sets
-    * Run as other user or system service
-    * Schedule bashbot from Cron
-    * Use from CLI and Scripts
-    * Customize Bashbot Environment
-* [Best Practices](doc/5_practice.md)
-    * Customize mycommands.sh
-    * Overwrite/disable commands
-    * Separate logic from commands
-    * Test your Bot with shellcheck
-* [Function Reference](doc/6_reference.md)
-    * Sending Messages, Files, Keyboards
-    * User Access Control
-    * Inline Queries
-    * jsshDB Bashbot key-value storage
-    * Background and Interactive Jobs
-* [Developer Notes](doc/7_develop.md)
-    * Debug bashbot
-    * Modules, addons, events
-    * Setup your environment
-    * Bashbot test suite
-* [Examples Dir](examples/README.md)
-
-### Your really first bashbot in a nutshell
-
-To install and run bashbot you need access to a linux/unix command line. If you 
-don't know how to get access to a linux/unix command line you should stop 
-reading here :-(
-
-In addition you need a [Telegram client](https://telegram.org) and a mobile 
-phone to [register an 
-account](https://telegramguide.com/create-a-telegram-account/).
-If you don't want to register for Telegram you should stop reading here ;-)
-
-After you're registered to Telegram send a message to 
-[@botfather](https://telegram.me/botfather),
-[create a new Telegram Bot token](doc/1_firstbot.md) and write it down. You 
-need the token to install the bot.
-
-Now open a terminal and check if bash is installed:
- ```
-which bash && echo "bash seems available..."
-```
+Documentation
 
 
-Create a new directory, change to it:  ```mkdir tbb; cd tbb``` and download the 
-latest '*.tar.gz' file from
-[https://github.com/topkecleon/telegram-bot-bash/releases](https://github.com/to
-pkecleon/telegram-bot-bash/releases). This can be done with the commands:
-```bash
-wget -q https://github.com/$(wget -q 
-https://github.com/topkecleon/telegram-bot-bash/releases/latest -O - | egrep 
-'/.*/.*/.*tar.gz' -o)
-```
+* Introduction to Telegram Bots [https://core.telegram.org/bots]
+* Install Bashbot [doc/0_install.md]
 
-Extract the '*.tar.gz' file and change to bashbot directory: ```tar -xzf 
-*.tar.gz; cd telegram-bot-bash```,
-install bashbot: ```./bashbot.sh init``` and enter your bot token when asked. 
-All other questions can be answered
-by hitting the \<Return\> key.
+  o Install release
+  o Install from github
+  o Update Bashbot
+  o Notes on Updates
 
-That's all, now you can start your bot with ```./bashbot.sh start``` and send 
-him messages:
-```
-/start
+* Get Bottoken from Botfather [doc/1_firstbot.md]
+* Getting Started [doc/2_usage.md]
 
-You are Botadmin
-*Available commands*:
-*• /start*: _Start bot and get this message_.
-*• /help*: _Get this message_.
-*• /info*: _Get shorter info message about this bot_....
+  o Managing your Bot
+  o Receive data
+  o Send messages
+  o Send files, locations, keyboards
 
-/info
+* Advanced Features [doc/3_advanced.md]
 
-This is bashbot, the Telegram bot written entirely in bash.
-It features background tasks and interactive chats, and can serve as an 
-interface for CLI programs.
-```
-For more Information on how to install, customize and use your new bot, read 
-the [Documentation](#Documentation)
+  o Access Control
+  o Interactive Chats
+  o Background Jobs
+  o Inline queries
+  o Send message errors
 
-### Log files
+* Expert Use [doc/4_expert.md]
 
-Since version 0.96 bashbot log commands received/send and connection errors. If 
-you start bashbot in debug mode
-bash stdout, stderr and all send/received telegram message are logged also.
+  o Handling UTF-8 character sets
+  o Run as other user or system service
+  o Schedule bashbot from Cron
+  o Use from CLI and Scripts
+  o Customize Bashbot Environment
 
-To enable debug mode start bashbot with debug as third argument: `bashbot start 
-debug`
+* Best Practices [doc/5_practice.md]
 
-```
-├── logs 
-│   ├── BASHBOT.log      # log what your bot is doing ...
-│   ├── ERROR.log        # connection errors from / to telegram API
-│   │
-│   ├── DEBUG.log        # stdout/stderr of you bot (debug mode enabled)
-│   └── MESSAGE.log      # full text of all message send/received (debug mode 
-enabled)
-```
+  o Customize mycommands.sh
+  o Overwrite/disable commands
+  o Separate logic from commands
+  o Test your Bot with shellcheck
 
-----
+* Function Reference [doc/6_reference.md]
 
-## Security Considerations
-Running a Telegram Bot means it is connected to the public and you never know 
-what's send to your Bot.
+  o Sending Messages, Files, Keyboards
+  o User Access Control
+  o Inline Queries
+  o jsshDB Bashbot key-value storage
+  o Background and Interactive Jobs
 
-Bash scripts in general are not designed to be bullet proof, so consider this 
-Bot as a proof of concept. Bash programmers often struggle with 'quoting hell' 
-and globbing, see [Implications of wrong 
-quoting](https://unix.stackexchange.com/questions/171346/security-implications-o
-f-forgetting-to-quote-a-variable-in-bash-posix-shells)
+* Developer Notes [doc/7_develop.md]
 
-Whenever you are processing input from untrusted sources (messages, files, 
-network) you must be as careful as possible, e.g. set IFS appropriate, disable 
-globbing (set -f) and quote everything. In addition delete unused scripts and 
-examples from your Bot, e.g. scripts 'notify', 'calc', 'question', and disable 
-all not used commands.
+  o Debug bashbot
+  o Modules, addons, events
+  o Setup your environment
+  o Bashbot test suite
 
-**Note:** Up to version v0.941 (mai/22/2020) telegram-bot-bash had a remote 
-code execution (RCE) bug, please update if you use an older version!
-see [Issue #125](https://github.com/topkecleon/telegram-bot-bash/issues/125)
+* Examples Directory [examples]
+* Webhook Example [examples/webhook]
 
-One of the most powerful features of unix shells is variable and command 
-substitution using ```${}``` and ```$()```,
-but as they are expanded in double quotes, this can lead to RCE and information 
-disclosing bugs in complex scripts like bashbot.
-So it's more secure to escape or remove '$' in input from user, files or 
-network.
 
-A powerful tool to improve your scripts is ```shellcheck```. You can [use it 
-online](https://www.shellcheck.net/) or [install shellcheck 
-locally](https://github.com/koalaman/shellcheck#installing). Shellcheck is used 
-extensively in bashbot development to ensure a high code quality, e.g. it's not 
-allowed to push changes without passing all shellcheck tests.
-In addition bashbot has a [test suite](doc/7_develop.md) to check if important 
+Your very first bashbot in a nutshell
+
+To install and run bashbot you need access to a Linux/Unix command line with bash, a
+Telegram client [https://telegram.org] and a mobile phone with a Telegram account [https:/
+/telegramguide.com/create-a-telegram-account/].
+First you need to create a new Telegram Bot token [doc/1_firstbot.md] for your bot and
+write it down.
+Now open a Linux/Unix terminal with bash, create a new directory, change to it and install
+telegram-bot-bash:
+
+  # create bot dir
+  mkdir mybot
+  cd mybot
+
+  # download latest release with wget or from https://github.com/topkecleon/telegram-bot-
+  bash/releases/latest
+  wget &quot;https://github.com/$(wget -q &quot;https://github.com/topkecleon/telegram-
+  bot-bash/releases/latest&quot; -O - | egrep '/.*/download/.*/.*tar.gz' -o)&quot;
+
+  # Extract the tar archive and go into bot dir
+  tar -xzf *.tar.gz
+  cd telegram-bot-bash
+
+  # initialize your bot
+  # Enter your bot token when asked, all other questions can be answered by hitting the
+  \<Return\> key.
+  ./bashbot.sh init
+
+  # Now start your bot
+  ./bashbot.sh start
+
+  Bottoken is valid ...
+  Bot Name: yourbotname_bot
+  Session Name: yourbotname_bot-startbot
+  Bot started successfully.
+
+Now open the Telegram App on your mobile phone and start a chat with your bot (your bot's
+username is shown after 'Bot Name:'):
+
+  /start
+
+  You are Botadmin
+  Available commands:
+    /start: _Start bot and get this message_.
+    /help: _Get this message_.
+    /info: _Get shorter info message about this bot_....
+
+  /info
+
+  This is bashbot, the Telegram bot written entirely in bash.
+  It features background tasks and interactive chats, and can serve as an interface for
+  CLI programs.
+
+For more Information on how to install, customize and use your new bot, read the
+Documentation [#Documentation].
+
+Log files
+
+Bashbot actions are logged to BASHBOT.log. Telegram send/receive errors are logged to
+ERROR.log. Start bashbot in debug mode to see all messages sent to / received from
+Telegram, as well as bash command error messages.
+To enable debug mode, start bashbot with debug as third argument: bashbot start debug
+
+  |__ logs
+  |     |__ BASHBOT.log  # log what your bot is doing ...
+  |     |__ ERROR.log    # connection errors from / to Telegram API
+  |     |
+  |     |__ DEBUG.log    # stdout/stderr of you bot (debug mode enabled)
+  |     |__ MESSAGE.log  # full text of all message send/received (debug mode enabled)
+
+------------------------------------------------------------------------------------------
+
+Security Considerations
+
+Running a Telegram Bot means it is connected to the public and you never know what's send
+to your Bot.
+Bash scripts in general are not designed to be bulletproof, so consider this Bot as a
+proof of concept. Bash programmers often struggle with 'quoting hell' and globbing, see
+Implications of wrong quoting [https://unix.stackexchange.com/questions/171346/security-
+implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells].
+Whenever you are processing input from untrusted sources (messages, files, network) you
+must be as careful as possible (e.g. set IFS appropriately, disable globbing with set -
+f and quote everything). In addition remove unused scripts and examples from your Bot
+(e.g. everything in example/) and disable/remove all unused bot commands.
+It's important to escape or remove $ and ` in input from user, files or network (as
+bashbot does). One of the powerful features of Unix shells is variable and command
+substitution using ${var}, $(cmd) and `cmd` can lead to remote code execution (RCE) or
+remote information disclosure (RID) bugs if unescaped $ or ` is included in untrusted
+input (e.g. $$ or $(rm -rf /*)).
+A powerful tool to improve your scripts is shellcheck. You can use it online [https://
+www.shellcheck.net/] or install shellcheck locally [https://github.com/koalaman/
+shellcheck#installing]. Shellcheck is used extensively in bashbot development to ensure a
+high code quality (e.g. it's not allowed to push changes without passing all shellcheck
+tests). In addition bashbot has a test suite [doc/7_develop.md] to check if important
 functionality is working as expected.
 
-### Use printf whenever possible
+Use printf whenever possible
 
-If you're writing a script and it is taking external input (from the user as 
-arguments or file system...),
-you shouldn't use echo to display it. [Use printf whenever 
-possible](https://unix.stackexchange.com/a/6581)
+If you're writing a script that accepts external input (e.g. from the user as arguments or
+the file system), you shouldn't use echo to display it. Use printf whenever possible
+[https://unix.stackexchange.com/a/6581].
 
-```bash
-  # very simple
-  echo "text with variables. PWD=$PWD"
-  printf '%s\n' "text with variables. PWD=$PWD"
-  -> text with variables. PWD=/home/xxx
+Run your Bot as a restricted user
 
-  # more advanced
-  FLOAT="1.2346777892864" INTEGER="12345.123"
-  echo "text with variabeles. float=$FLOAT, integer=$INTEGER, PWD=$PWD"
-  ->text with variables. float=1.2346777892864, integer=12345.123, PWD=/home/xxx
+I recommend running your bot as a user with almost no access rights. All files your Bot
+has write access to are in danger of being overwritten/deleted if your bot is hacked. For
+the same reason every file your Bot can read is in danger of being disclosed. Restrict
+your Bots access rights to the absolute minimum.
+Never run your Bot as root, this is the most dangerous you can do! Usually the user
+'nobody' has almost no rights on Linux/Unix systems. See Expert use [doc/4_expert.md] on
+how to run your Bot as an other user.
 
-  printf "text with variables. float=%.2f, integer=%d, PWD=%s\n" "" "$INTEGER" 
-"$PWD"
-  ->text with variables. float=1.23, integer=12345, PWD=/home/xxx
-```
+Secure your Bot installation
 
-### Do not use #!/usr/bin/env bash
+Your Bot configuration must not be readable by other users. Everyone who can read your
+Bots token is able to act as your Bot and has access to all chats the Bot is in!
+Everyone with read access to your Bot files can extract your Bots data. Especially your
+Bot config in config.jssh must be protected against other users. No one except you should
+have write access to the Bot files. The Bot should be restricted to have write access to
+count.jssh, data-bot-bash/ and logs/ only, all other files must be write protected.
+To set access rights for your bashbot installation to a reasonable default run sudo ./
+bashbot.sh init after every update or change to your installation directory.
+Note: Keep old log files in a safe place or even better delete them, they are GDPR
+relevant and may contain information [https://github.com/topkecleon/telegram-bot-bash/
+issues/174] you don't want to be public.
 
-**We stay with /bin/bash shebang, because it's more save from security 
-perspective.**
-
-Use of a fixed path to the system provided bash makes it harder for attackers 
-or users to place alternative versions of bash
-and avoids using a possibly broken, mangled or compromised bash executable. 
-
-If you are a BSD /  MacOS user or must to use an other bash location, see 
-[Install Bashbot](doc/0_install.md)
-
-### Run your Bot as a restricted user
-**I recommend to run your bot as a user, with almost no access rights.** 
-All files your Bot have write access to are in danger to be overwritten/deleted 
-if your bot is hacked.
-For the same reason every file your Bot can read is in danger to be disclosed. 
-Restrict your Bots access rights to the absolute minimum.
-
-**Never run your Bot as root, this is the most dangerous you can do!** Usually 
-the user 'nobody' has almost no rights on unix/linux systems. See [Expert 
-use](doc/4_expert.md) on how to run your Bot as an other user.
-
-### Secure your Bot installation
-**Your Bot configuration must no be readable from other users.** Everyone who 
-can read your Bots token is able to act as your Bot and has access to all chats 
-the Bot is in!
-
-Everyone with read access to your Bot files can extract your Bots data. 
-Especially your Bot config in ```config.jssh``` must be protected against other 
-users. No one except you should have write access to the Bot files. The Bot 
-should be restricted to have write access to ```count.jssh``` and  
-```data-bot-bash``` only, all other files must be write protected.
-
-To set access rights for your bashbot installation to a reasonable default run 
-```sudo ./bashbot.sh init``` after every update or change to your installation 
-directory.
-
-## FAQ
-
-### Is this Bot insecure?
-Bashbot is not more (in)secure as any Bot written in an other language, we have 
-done our best to make it as secure as possible. But YOU are responsible for the 
-bot commands you wrote and you should know about the risks ...
-
-**Note:** Up to version 0.941 (mai/22/2020) telegram-bot-bash had a remote code 
-execution bug, please update if you use an older version!
-
-### Why Bash and not the much better xyz?
-Well, that's a damn good question ... may be because I'm an unix admin from 
-stone age. Nevertheless there are more reasons from my side:
-
-- bashbot will run everywhere where bash and (gnu) sed is available, from 
-embedded linux to mainframe
-- easy to integrate with other shell script, e.g. for sending system message / 
-health status
-- no need to install or learn a new programming language, library or framework
-- no database, not event driven, not object oriented ...
-
-### Can I have the single bashbot.sh file back?
-At the beginning bashbot was simply the file ```bashbot.sh``` you can copy 
-everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 
-'modules/*.sh' and much more.
-
-Hey no Problem, if you are finished with your cool bot run 
-```dev/make-standalone.sh``` to create a stripped down Version of your bot 
-containing only
-'bashbot.sh' and 'commands.sh'! For more information see [Create a stripped 
-down Version of your Bot](doc/7_develop.md)
-
-### Can I send messages from CLI and scripts?
-Of course, you can send messages from CLI and scripts, simply install bashbot 
-as [described here](#Your-really-first-bashbot-in-a-nutshell),
-send the message '/start' to set yourself as botadmin and stop the bot with 
-```./bashbot.sh stop```.
-
-Run the following commands in your bash shell or script while you are in the 
-installation directory:
-
-```bash
-# prepare bash / script to send commands
-export BASHBOT_HOME="$(pwd)"
-source ./bashbot.sh source
-
-# send me a test message
-send_message "$(getConfigKey "botadmin")" "test"
-
-# send me output of a system command
-send_message "$(getConfigKey "botadmin")" "$(df -h)"
-```
-For more information see [Expert Use](doc/8_custom.md)
+FAQ
 
 
-### Blocked by telegram?
-This may happen if to many or wrong requests are sent to api.telegram.org, e.g. 
-using a invalid token or not existing API calls.
-If the block stay for longer time you can ask telegram service to unblock your 
-IP-Adress. 
+Is this Bot insecure?
 
+Bashbot is not more (in)secure than a Bot written in another language. We have done our
+best to make it as secure as possible. But YOU are responsible for the bot commands you
+wrote and you should know about the risks ...
+Note: Up to version 0.941 (mai/22/2020) telegram-bot-bash had a remote code execution bug,
+please update if you use an older version!
+
+Why Bash and not the much better xyz?
+
+Well, that's a damn good question... maybe because I'm a Unix admin from the stone age.
+Nevertheless there are more reasons from my side:
+
+* bashbot will run wherever bash and (gnu) sed is available, from embedded Linux to
+  mainframe
+* easy to integrate with other shell scripts, e.g. for sending system message / health
+  status
+* no need to install or learn a new programming language, library or framework
+* no database, not event driven, not object oriented ...
+
+
+Can I have the single bashbot.sh file back?
+
+At the beginning bashbot was simply the file bashbot.sh that you could copy everywhere and
+run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.
+Hey no problem, if you are finished with your cool bot, run dev/make-standalone.sh to
+create a stripped down version of your bot containing only 'bashbot.sh' and 'commands.sh'!
+For more information see Create a stripped down version of your Bot [doc/7_develop.md].
+
+Can I send messages from CLI and scripts?
+
+Of course you can send messages from command line and scripts! Simply install bashbot as
+described here [#Your-really-first-bashbot-in-a-nutshell], send the message '/start' to
+set yourself as botadmin and then stop the bot with ./bashbot.sh stop.
+Bashbot provides some ready to use scripts for sending messages from command line in bin/
+dir, e.g. send_message.sh.
+
+  bin/send_message.sh BOTADMIN &quot;This is my first message send from CLI&quot;
+
+  bin/send_message.sh --help
+
+You can also source bashbot for use in your scripts, for more information see Expert Use
+[doc/4_expert.md].
+
+Blocked by telegram?
+
+This may happen if too many or wrong requests are sent to api.telegram.org, e.g. using a
+invalid token or invalid API calls. If the block stay for longer time you can ask telegram
+service to unblock your IP-Address.
 You can check with curl or wget if you are blocked by Telegram:
-```bash
-curl -m 10  https://api.telegram.org/bot
-#curl: (28) Connection timed out after 10001 milliseconds
 
-wget -t 1 -T 10 https://api.telegram.org/bot
-#Connecting to api.telegram.org (api.telegram.org)|46.38.243.234|:443... 
-failed: Connection timed out.
-```
+  curl -m 10  https://api.telegram.org/bot
+  #curl: (28) Connection timed out after 10001 milliseconds
 
-Since Version 0.96 bashbot offers the option to recover from broken connections 
-(aka blocked). Therefore you can provide a function
-named `bashbotBlockRecover()` in `mycommands.sh`. If the function exists it is 
-called every time when a broken connection is detected.
+  wget -t 1 -T 10 https://api.telegram.org/bot
+  #Connecting to api.telegram.org (api.telegram.org)|46.38.243.234|:443... failed:
+  Connection timed out.
 
-Possible actions are: Check if network is working, change IP-Adress or simply 
-wait some time.
-
-If everything seems OK return 0 for retry or any non 0 value to give up.
-
-```bash
-# called when bashbot sedn command failed because we can not connect to telegram
-# return 0 to retry, return non 0 to give up
-bashbotBlockRecover() {
-	# place your commands to unblock here, e.g. change IP-Adess or simply 
-wait
-	sleep 60 && return 0 # may be temporary
-	return 1 
-    }
-
-```
- 
+  nc -w 2 api.telegram.org 443 || echo &quot;your IP seems blocked by telegram&quot;
+  #your IP seems blocked by telegram
 
+Bashbot offers the option to recover from broken connections (blocked). Therefore you can
+provide a function named bashbotBlockRecover() in mycommands.sh, the function is called
+every time when a broken connection is detected.
+Possible actions are: Check if network is working, change IP-Address or simply wait some
+time. See mycommnds.sh.dist for an example.
+------------------------------------------------------------------------------------------
 @Gnadelwartz
 
-## That's it all guys!
+That's it all guys!
 
-If you feel that there's something missing or if you found a bug, feel free to 
-submit a pull request!
+If you feel that there's something missing or if you found a bug, feel free to submit a
+pull request!
+
+$$VERSION$$ v1.52-0-g1a83202
 
-#### $$VERSION$$ v1.1-0-ge835bbc

addons/antiFlood.sh

@@ -4,7 +4,7 @@
 # this addon counts how many files, e.g. stickers, are sent to
 # a chat and takes actions if threshold is reached
 #  
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # used events:
 #
@@ -37,16 +37,16 @@ ANTIFL_BAN="5"	# 5 minutes
 
 # initialize after installation or update
 if [[ "$1" = "init"* ]]; then 
-	jssh_newDB "addons/$ANTIFL_ME"
+	jssh_newDB "addons/${ANTIFL_ME}"
 fi
 
 
 
 # register on startbot
 if [[ "$1" = "start"* ]]; then 
-    ANTIFL_ADMIN="$(< "${BOTADMIN}")"
+    ANTIFL_ADMIN="$(getConfigKey "botadmin")"
     #load existing chat settings on start
-    jssh_readDB "ANTIFL_CHATS" "addons/$ANTIFL_ME"
+    jssh_readDB "ANTIFL_CHATS" "addons/${ANTIFL_ME}"
 
     # register to CMD
     BASHBOT_EVENT_CMD["${ANTIFL_ME}"]="${ANTIFL_ME}_cmd"
@@ -71,14 +71,14 @@ if [[ "$1" = "start"* ]]; then
 		"/afdo" | "/afactive")
 			[[ "${CMD[1]}"  =~ ^[-0-9]+$  ]] && user_is_botadmin "${USER[ID]}" && chat="$3"
 			ANTIFL_CHATS["${chat}","active"]="yes"
-			jssh_writeDB "ANTIFL_CHATS" "addons/$ANTIFL_ME" &
+			jssh_writeDB "ANTIFL_CHATS" "addons/${ANTIFL_ME}" &
 			send_normal_message "${USER[ID]}" "Antiflood activated for chat ${chat}" &
 		;;
 		# command /afactive starts counter meausares
 		"/afstop")
 			[[ "${CMD[1]}"  =~ ^[-0-9]+$  ]] && user_is_botadmin "${USER[ID]}" && chat="$3"
 			ANTIFL_CHATS["${chat}","active"]="no"
-			jssh_writeDB "ANTIFL_CHATS" "addons/$ANTIFL_ME" &
+			jssh_writeDB "ANTIFL_CHATS" "addons/${ANTIFL_ME}" &
 			send_normal_message "${USER[ID]}" "Antiflood stopped for chat ${chat}" &
 		;;
 	esac
@@ -90,7 +90,7 @@ if [[ "$1" = "start"* ]]; then
     # save settings and reset flood level every BAN Min
     antiFlood_timer(){
 	ANTIFL_ACTUALS=( ) 
-	jssh_writeDB "ANTIFL_CHATS" "addons/$ANTIFL_ME" &
+	jssh_writeDB "ANTIFL_CHATS" "addons/${ANTIFL_ME}" &
     }
 
     # register to inline and command

addons/example.sh

@@ -4,7 +4,7 @@
 # Addons can register to bashbot events at startup
 # by providing their name and a callback per event
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 #
 # If an event occurs each registered event function is called.
 #
@@ -60,6 +60,7 @@ fi
 if [[ "$1" = "start"* ]]; then 
     # register to reply
     BASHBOT_EVENT_REPLY["${EXAMPLE_ME}"]="${EXAMPLE_ME}_reply"
+    EXAMPLE_ADMIN="$(getConfigKey "botadmin")"
 
     # any function defined by addons MUST be prefixed by addon name
     # function local variables can have any name, but must be LOCAL
@@ -87,7 +88,7 @@ if [[ "$1" = "start"* ]]; then
     # any function defined by addons MUST be prefixed by addon name
     # function local variables can have any name, but must be LOCAL
     example_after5min(){
-	send_markdown_message "$(< "${BOTADMIN}")" "This is a one time event after 5 Minutes!" &
+	send_markdown_message "${EXAMPLE_ADMIN}" "This is a one time event after 5 Minutes!" &
     }
 
     BASHBOT_EVENT_TIMER["${EXAMPLE_ME}every2min","2"]="${EXAMPLE_ME}_every2min"
@@ -95,7 +96,7 @@ if [[ "$1" = "start"* ]]; then
     # any function defined by addons MUST be prefixed by addon name
     # function local variables can have any name, but must be LOCAL
     example_every2min(){
-	send_markdown_message "$(< "${BOTADMIN}")" "This a a every 2 minute event ..." &
+	send_markdown_message "${EXAMPLE_ADMIN}" "This a a every 2 minute event ..." &
     }
 
     # register to send
@@ -109,6 +110,6 @@ if [[ "$1" = "start"* ]]; then
     # Note: do not call any send message functions from EVENT_SEND!
     example_log(){
 	local send="$1"; shift
-	echo "$(date): Type: ${send} Args: $*" >>"${EXAMPLE_LOG}"
+	printf "%s: Type: %s Args: %s\n" "$(date)" "${send}" "$*" >>"${EXAMPLE_LOG}"
     }
 fi

bashbot.rc

@@ -1,9 +1,14 @@
-#!/bin/sh
+#!/bin/bash
 # description: Start or stop telegram-bash-bot
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+# example service script to run bashbot in background as specified user
+#
+# tested on: ubuntu, opensuse, debian
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
 # shellcheck disable=SC2009
 # shellcheck disable=SC2181
+# shellcheck disable=SC2250
 
 #
 ### BEGIN INIT INFO
@@ -23,60 +28,127 @@ runcmd="echo Dry run:" # not activated until you edit lines below
 # Configuration Section
 
 # edit the next line to fit the user you want to run bashbot, e.g. nobody:
-runas="nobody" 
+runas="nobody"
 
-# uncomment one of the following lines to fit your system 
-# runcmd="su $runas -s /bin/bash -c "      # runasuser with *su*
-# runcmd="runuser $runas -s /bin/bash -c " # runasuser with *runuser*
+# uncomment one of the example lines to fit your system 
+# runcmd="su ${runas} -s /bin/bash -c "      # runasuser with *su*
+# runcmd="/usr/sbin/runuser ${runas} -s /bin/bash -c " # runasuser with *runuser*
 
 # edit the values of the following lines to fit your config:
-start="cd /usr/local/telegram-bot-bash; /usr/local/telegram-bot-bash/bashbot.sh"	# location of your bashbot.sh script
-name='' 	# your bot name as given to botfather, e.g. mysomething_bot
+# your bot name as given to botfather, e.g. mysomething_bot
+name=""
+[ -z "${name}" ] && name="unknown"
+
+# your bot installation dir
+bashbotdir="/usr/local/telegram-bot-bash"
+databotdir="${bashbotdir}/data-bot-bash"
+FIFO="${databotdir}/webhook-fifo-${name}"
+
+# programs to run
+bashbot="cd ${bashbotdir}; ${bashbotdir}/bashbot.sh"
+webhook="cd ${bashbotdir}; nohup ${bashbotdir}/bin/process_batch.sh --startbot --watch ${FIFO}"
+# set additionl parameter, e.g. debug
+mode=""
+
+# select logfile for webhook start stop and script errors
+hooklog="DEBUG"
+hooklog="WEBHOOK"
 
 # END Configuration
 #######################
 
-[ "$name" = "" ] && name="$runas"
+
+# check for bot status
+stat=""
+ps -f -u "${runas}" | grep "${name}" | grep -qF "bashbot.sh startbot"
+if [ "$?" = "0" ]; then
+	# printf "bashbot (%s) is running in poll mode\n" "${name}"
+	stat="${stat} polling"
+fi
+ps -f -u "${runas}" | grep "${name}" | grep -qF "process_batch.sh --startbot"
+if [ "$?" = "0" ]; then
+	#printf "bashbot (%s) is running in webhook mode\n" "${name}"
+	stat="${stat} webhook"
+elif [ "${name}" != "unknown" ]; then
+	#printf "bashbot (%s) is stopped\n" "${name}"
+	stat="stop"
+else
+	stat="unknown"
+fi
 
 case "$1" in
 'start')
-	$runcmd "$start start" # >/dev/null 2>&1 </dev/null
+	[ "${stat}" != "stop" ] && printf "Warning, bot is already running in mode: %s\n" "${stat}"
+	$runcmd "$bashbot start $mode" # >/dev/null 2>&1 </dev/null
+	RETVAL=$?
+	;;
+'starthook')
+	[ -p "${FIFO}" ] || printf "Warning, webhook pipe not found: %s\n" "${FIFO##*/}"
+	[ "${stat}" != "stop" ] && printf "Warning, bot is already running in mode: %s\n" "${stat}"
+	printf "Starting bashbot in webhook mode ... "
+	$runcmd "$webhook $mode </dev/null &>>${bashbotdir}/logs/${hooklog}.log &" # >/dev/null 2>&1 </dev/null
+	sleep 1
+	$0 status
 	RETVAL=$?
 	;;
 'stop')
-	$runcmd "$start stop"
+	[[ "${stat}" != *"poll"* ]] && printf "Warning, bot is not in poll mode: %s\n" "${stat}"
+	$runcmd "$bashbot stop $mode"
 	RETVAL=$?
 	;;
-'status')
-	ps -f -u "$runas" | grep "$name" | grep -qF "bashbot.sh startbot"
-	if [ "$?" = "0" ]; then
-		echo "bashbot ($name) is running"
-		RETVAL=0
-	else
-		echo "bashbot ($name) is stopped"
-		RETVAL=1
+'stophook')
+	[[ "${stat}" != *"hook"* ]] && printf "Warning, bot is not in webhook mode: %s\n" "${stat}"
+	printf "Stopping bashbot webhook mode ... "
+	KILLID="$(ps -f -u "${runas}" | grep "process_batch.sh --startbot" | sed -E 's/[^0-9]+([0-9]+).*/\1/' | tr -s "\r\n" "  ")"
+	if [ -n "${KILLID}" ]; then
+		$runcmd "kill ${KILLID} 2>/dev/null; wait ${KILLID} 2>/dev/null"
+		sleep 1
 	fi
+	RETVAL=$?
+	$0 status
+	;;
+'status')
+	case "${stat}" in
+		*"poll"*) printf "bashbot (%s) is running in polling mode\n" "${name}"
+			RETVAL=0
+			;;&
+		*"hook"*) printf "bashbot (%s) is running in webhook mode\n" "${name}"
+			RETVAL=0
+			;;
+		*"stop"*) printf "bashbot (%s) is not running\n" "${name}"
+			RETVAL=1
+			;;
+		*)	printf "bashbot (%s) status is %s\n" "${name}" "${stat}"
+			RETVAL=2
+			;;
+	esac
 	;;
 'restart'|'reload')
 	$0 stop; $0 start 
 	RETVAL=$?
 	;;
+'restarthook'|'reloadhook')
+	$0 stophook; $0 starthook 
+	RETVAL=$?
+	;;
 'restartback')
 	$0 suspendback; $0 resumeback 
 	RETVAL=$?
 	;;
 'suspendback'|'resumeback'|'killback')
-	$runcmd "$start $1"
+	# shellcheck disable=SC2250
+	$runcmd "$bashbot $1"
 	RETVAL=$?
 	# kill inotifywait from runuser
 	if [ "$1" != "resumeback" ]; then 
 		# shellcheck disable=SC2046
-		kill -9 $(ps -u "$runas" | grep inotifywait | sed 's/ .*//') >/dev/null 2>&1
+		kill -9 $(ps -u "${runas}" | grep inotifywait | sed 's/ .*//') >/dev/null 2>&1
 	fi  
 	;;
 *)
-	echo "Usage: $0 { start | stop | restart | reload | restartback | suspendback | resumeback | killback }"
+	printf "%s\n" "Usage: $0 [ start  | stop | restart | starthook |  stophook  | restarthook ]"
+	printf "%s\n" "       $0 [ status | restartback | suspendback  | resumeback | killback ]"
 	RETVAL=1
 	;;
 esac
-exit $RETVAL
+exit "${RETVAL}"

bin/any_command.sh

@@ -0,0 +1,85 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034,SC2059
+#===============================================================================
+#
+#          FILE: bin/any_command.sh
+#
+USAGE='any_command.sh [-h|--help] [--force|--reference] bot_command args ...'
+# 
+#   DESCRIPTION: execute (almost) any bashbot command/function
+#                can be used for testing commands while bot development
+# 
+#       OPTIONS: -- force - execute unknown commands/functions
+#		      by default only commands in 6_reference.md are allowed
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 30.01.2021 10:24
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+COMMAND=""
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "debug" # debug
+print_help "$1"
+
+
+error=""
+# check options
+if [[ "$1" = "--force"  ]]; then
+	# skip checks
+	shift
+else
+	# check for --ref 
+	ref="$1"; [[ "$1" == "--ref"* ]] && shift
+	if [ "${#1}" -lt  11 ];then
+		printf "${RED}Command must be minimum 11 characters!${NC}\n"
+		error=3
+	fi
+	if [[ "$1" != *"_"* ]];then
+		printf "${RED}Command must contain _ (underscore)!${NC}\n"
+		error=3
+	fi
+	# simple hack to get allowed commands from doc
+	if grep -q "^##### $1" <<<"$(sed -n -e '/^##### _is_/,$ d' -e '/^##### /p' "${BASHBOT_HOME:-..}doc/"6_*)"; then
+		# oiutput reference and exit
+		if [[ "${ref}" == "--ref"* ]]; then
+			sed -n -e '/^##### '"$1"'/,/^##/ p' "${BASHBOT_HOME:-..}doc/"6_*
+			exit
+		fi
+	else
+		printf "Command ${GREY}%s${NC} not found in 6_reference.md, use ${GREY}--force${NC} to execute!\n" "$1"
+		error=4
+	fi
+	[ -n "${error}" ] && exit "${error}"
+fi
+
+
+####
+# ready, do stuff here -----
+COMMAND="$1"
+if [ "$2" == "BOTADMIN" ]; then
+	ARG1="${BOTADMIN}"
+else
+	ARG1="$2"
+fi
+
+# clear result and response
+BOTSENT=()
+UPD=()
+
+# send message in selected format
+"${COMMAND}" "${ARG1}"  "${@:3}"
+
+# output result an telegram response
+print_result
+print_response

bin/bashbot_env.inc.sh

@@ -0,0 +1,91 @@
+#!/bin/bash
+#===============================================================================
+#
+#          FILE: bashbot_env.inc.sh
+# 
+#         USAGE: source bashbot_env.inc.sh [debug]
+#
+#   DESCRIPTION: set bashbot environment for all scripts in this directory
+# 
+#       OPTIONS: $1 - will be forwarded ro bashbot, e.g. debug
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 18.12.2020 12:27
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+############
+# set where your bashbot lives
+export BASHBOT_HOME BASHBOT_ETC BASHBOT_VAR FILE_REGEX ME
+
+# default: one dir up 
+BASHBOT_HOME="$(cd "${BASH_SOURCE[0]%/*}/../" >/dev/null 2>&1 && pwd)"
+[ "${BASHBOT_HOME}" = "" ] && BASHBOT_HOME="../"
+
+# set you own BASHBOT_HOME if different, e.g.
+# BASHBOT_HOME="/usr/local/telegram-bot-bash"
+BASHBOT_VAR="${BASHBOT_HOME}"
+BASHBOT_ETC="${BASHBOT_HOME}"
+
+#####
+# if files are not readable, eviroment is wrong or bashbot is not initialized
+
+# check for bashbot
+if [ ! -r "${BASHBOT_HOME}/bashbot.sh" ]; then
+	printf "%s\n" "Bashbot.sh not found in \"${BASHBOT_HOME}\""
+	exit 4
+fi
+
+dev=" Are we in dev or did you forget to run init?"
+# check for botconfig.jssh readable
+if [ ! -r "${BASHBOT_ETC}/botconfig.jssh" ]; then
+	printf "%s\n" "Bashbot config file in \"${BASHBOT_ETC}\" does not exist or is not readable. ${dev}"
+	exit 3
+fi
+# check for count.jssh readable
+if [ ! -r "${BASHBOT_VAR}/count.jssh" ]; then
+	printf "%s\n" "Bashbot count file in \"${BASHBOT_VAR}\" does not exist or is not readable.  ${dev}"
+	exit 3
+fi
+
+# shellcheck disable=SC1090
+source "${BASHBOT_HOME}/bashbot.sh" source "$1"
+
+# overwrite bot FILE regex to BASHBOT_VAR
+# change this to the location you want to allow file uploads from
+UPLOADDIR="${BASHBOT_VAR%/bin*}"
+FILE_REGEX="${UPLOADDIR}/.*"
+
+# get and check ADMIN and NAME
+BOTNAME="$(getConfigKey "botname")"
+ME="${BOTNAME}"
+[[ -z "${BOTADMIN}" || "${BOTADMIN}" == "?" ]] && printf "%s\n" "${ORANGE}Warning: Botadmin not set, send bot command${NC} /start"
+[[ -z "${BOTNAME}"  ]] && printf "%s\n" "${ORANGE}Warning: Botname not set, run bashbot.sh botname"
+
+# default webhook pipe
+export WEBHOOK="${DATADIR}/webhook-fifo-${ME}"
+
+
+# output command result or Telegram response
+print_result() { jssh_printDB "BOTSENT" | sort -r; }
+print_response() { jssh_printDB "UPD"; }
+
+# check and output help
+print_help() {
+ case "$1" in
+	'')
+		printf "missing arguments\n"
+		;&
+	"-h"*)
+		printf 'usage: %s\n' "${USAGE}"
+		exit 1
+		;;
+	'--h'*)
+		sed -n '/^#====/,/^#====/p' <"$0"
+		exit 1
+		;;
+ esac
+}
+

bin/bashbot_init.inc.sh

@@ -0,0 +1,149 @@
+#!/bin/bash
+#===============================================================================
+#
+#          FILE: bashbot_init.inc.sh
+# 
+#         USAGE: source bashbot_init.inc.sh
+#
+#   DESCRIPTION: extend / overwrite bashbot initialisation
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 27.01.2021 13:42
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+# shellcheck disable=SC2059
+
+##########
+# commands to execute before bot_init() is called
+
+
+########
+# called after default init is finished
+my_init() {
+	: # your init here
+}
+
+
+#########
+#
+# extended initialisation:
+#
+# - uograde old config
+# - backup of botconfig.jssh
+# - running bot as service or other user
+# - copy clean and dist files if not exist
+# - configure bot for INLINE CALLBACK MEONLY SILENCER
+# 
+# delete from here to disable extended initialisation
+bot_init() {
+	if [ -n "${BASHBOT_HOME}" ] && ! cd "${BASHBOT_HOME}"; then
+		 printf "Can't change to BASHBOT_HOME"
+		 exit 1
+	fi
+	local runuser chown touser botname DEBUG="$1"
+	# upgrade from old version
+	# currently no action
+	printf "Check for Update actions ...\n"
+	printf "Done.\n"
+	# load addons on startup
+	printf "Initialize modules and addons ...\n"
+	for addons in "${ADDONDIR:-.}"/*.sh ; do
+		# shellcheck source=./modules/aliases.sh
+		[ -r "${addons}" ] && source "${addons}" "init" "${DEBUG}"
+	done
+	printf "Done.\n"
+	# guess bashbot from botconfig.jssh owner:group
+	[ -f "${BOTCONFIG}.jssh" ] && runuser="$(stat -c '%U' "${BOTCONFIG}.jssh"):$(stat -c '%G' "${BOTCONFIG}.jssh")"
+	# empty or ":" use user running init, nobody for root
+	if [ "${#runuser}" -lt 3 ]; then
+		# shellcheck disable=SC2153
+		runuser="${RUNUSER}"
+		[ "${UID}" = "0" ] && runuser="nobody"
+	fi
+	printf "Enter User to run bashbot [${runuser}]: "
+	read -r chown
+	[ -z "${chown}" ] && chown="${runuser}"
+	touser="${chown%:*}"
+	# check user ...
+	if ! id "${touser}" &>/dev/null; then
+		printf "${RED}User \"${touser}\" does not exist!${NN}"
+		exit 3
+	elif [ "${UID}" != "0" ]; then
+		# different user but not root ...
+		printf "${ORANGE}You are not root, adjusting permissions may fail. Try \"sudo ./bashbot.sh init\"${NN}Press <CTRL+C> to stop or <Enter> to continue..." 1>&2
+		[ -n "${INTERACTIVE}" ] && read -r runuser
+	fi
+	# check if mycommands exist
+	if [[ ! -r "${BASHBOT_ETC:-.}/mycommands.sh" && -r ${BASHBOT_ETC:-.}/mycommands.sh.dist ]]; then
+		printf "Mycommands.sh not found, copy ${GREY}<C>lean file, <E>xamples or <N>one${NC} to mycommands.sh? (c/e/N) N\b"
+		read -r ANSWER
+		[[ "${ANSWER}" =~ ^[cC] ]] && cp -f "${BASHBOT_ETC:-.}/mycommands.sh.clean" "${BASHBOT_ETC:-.}/mycommands.sh"
+		[[ "${ANSWER}" =~ ^[eE] ]] && cp -f "${BASHBOT_ETC:-.}/mycommands.sh.dist" "${BASHBOT_ETC:-.}/mycommands.sh"
+		# offer to copy config also
+		if [ ! -r "${BASHBOT_ETC:-.}/mycommands.conf" ]; then
+			printf "Mycommands config file not found, copy ${GREY}mycommands.conf.dist${NC} to mycommands.conf? (Y/n) Y\b"
+			read -r ANSWER
+			[[ "${ANSWER}" =~ ^[nN] ]] || cp -f "${BASHBOT_ETC:-.}/mycommands.conf.dist" "${BASHBOT_ETC:-.}/mycommands.conf"
+		fi
+		# adjust INLINE CALLBACK MEONLY SILENCER
+		if [ -w "${BASHBOT_ETC:-.}/mycommands.conf" ]; then
+			printf "Activate processing for ${GREY}<I>nline queries, <C>allback buttons, <B>oth or <N>one${NC} in mycommands.sh? (i/c/b/N) N\b"
+			read -r ANSWER
+			[[ "${ANSWER}" =~ ^[iIbB] ]] && sed -i '/INLINE="/ s/^.*$/export INLINE="1"/' "${BASHBOT_ETC:-.}/mycommands.conf"
+			[[ "${ANSWER}" =~ ^[cCbB] ]] && sed -i '/CALLBACK="/ s/^.*$/export CALLBACK="1"/' "${BASHBOT_ETC:-.}/mycommands.conf"
+			printf "Always ignore commands for other Bots in chat ${GREY}(/cmd@other_bot)${NC}? (y/N) N\b"
+			read -r ANSWER
+			[[ "${ANSWER}" =~ ^[yY] ]] && sed -i '/MEONLY="/ s/^.*$/export MEONLY="1"/' "${BASHBOT_ETC:-.}/mycommands.conf"
+			printf "Delete administrative messages in chats ${GREY}(pinned, user join/leave, ...)${NC}? (y/N) N\b"
+			read -r ANSWER
+			[[ "${ANSWER}" =~ ^[yY] ]] && sed -i '/SILENCER="/ s/^.*$/export SILENCER="yes"/' "${BASHBOT_ETC:-.}/mycommands.conf"
+		fi
+		printf "Done.\n"
+	fi
+	# adjust permissions
+	printf "Adjusting files and permissions for user \"${touser}\" ...\n"
+	chown -Rf "${chown}" . ./*
+	chmod 711 .
+	chmod -R o-w ./*
+	chmod -R u+w "${COUNTFILE}"* "${BLOCKEDFILE}"* "${DATADIR}" logs "${LOGDIR}/"*.log 2>/dev/null
+	chmod -R o-r,o-w "${COUNTFILE}"* "${BLOCKEDFILE}"* "${DATADIR}" "${BOTACL}" 2>/dev/null
+	# jsshDB must writeable by owner
+	find . -name '*.jssh*' -exec chmod u+w \{\} +
+	printf "Done.\n"
+	# adjust values in bashbot.rc
+	if [ -w "bashbot.rc" ]; then
+		printf "Adjust user and botname in bashbot.rc ...\n"
+		sed -i '/^[# ]*runas=/ s|runas=.*$|runas="'"${touser}"'"|' "bashbot.rc"
+		sed -i '/^[# ]*bashbotdir=/ s|bashbotdir=.*$|bashbotdir="'"${PWD}"'"|' "bashbot.rc"
+		botname="$(getConfigKey "botname")"
+		[ -n "${botname}" ] && sed -i '/^[# ]*name=/ s|name=.*$|name="'"${botname}"'"|' "bashbot.rc"
+		printf "Done.\n"
+	fi
+	# ask to check bottoken online
+	if [ -z "$(getConfigKey "botid")" ]; then
+		printf "Seems to be your first init. Should I verify your bot token online? (y/N) N\b"
+		read -r ANSWER
+		if [[ "${ANSWER}" =~ ^[Yy] ]]; then
+			printf "${GREEN}Contacting telegram to verify your bot token ...${NN}"
+			$0 botname
+		fi 
+	fi
+	# check if botconf seems valid
+	printf "${GREEN}This is your bot config:${NN}${GREY}"
+	sed 's/^/\t/' "${BOTCONFIG}.jssh" | grep -vF '["bot_config_key"]'; printf "${NC}"
+	if check_token "$(getConfigKey "bottoken")" && [[ "$(getConfigKey "botadmin")" =~ ^[${o9o9o9}]+$ ]]; then
+		printf "Bot config seems to be valid. Should I make a backup copy? (Y/n) Y\b"
+		read -r ANSWER
+		if [[ -z "${ANSWER}" || "${ANSWER}" =~ ^[^Nn] ]]; then
+			printf "Copy bot config to ${BOTCONFIG}.jssh.ok ...\n"
+			cp "${BOTCONFIG}.jssh" "${BOTCONFIG}.jssh.ok"
+		fi 
+	else
+		printf "${ORANGE}Bot config may incomplete, pls check.${NN}"
+	fi
+	# show result
+	printf  "${GREY}"; ls -ldp "${DATADIR}" "${LOGDIR}" ./*.jssh* ./*.sh ./*.conf 2>/dev/null; printf "${NC}"
+	_exec_if_function my_init
+}

bin/bashbot_stats.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/bashbot_stats.sh
+# 
+USAGE='bashbot_stats.sh [-h|--help] [debug]'
+# 
+#   DESCRIPTION: output bashbot user stats
+# 
+#       OPTIONS: -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 23.12.2020 20:34
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "$1"
+[ -n "$1" ] && print_help "$1"
+
+####
+# ready, do stuff here -----
+
+echo -e "${GREEN}Hi I'm ${BOTNAME}.${NC}"
+declare -A STATS
+jssh_readDB_async "STATS" "${COUNTFILE}"
+for MSG in ${!STATS[*]}
+do
+	[[ ! "${MSG}" =~ ^[0-9-]*$ ]] && continue
+	(( USERS++ ))
+done
+for MSG in ${STATS[*]}
+do
+	(( MESSAGES+=MSG ))
+done
+if [ "${USERS}" != "" ]; then
+	echo -e "${GREY}A total of ${NC}${MESSAGES}${GREY} messages from ${NC}${USERS}${GREY} users are processed.${NC}"
+else
+	echo -e "${ORANGE}No one used your bot so far ...${NC}"
+fi
+jssh_readDB_async "STATS" "${BLOCKEDFILE}"
+for MSG in ${!STATS[*]}
+do
+	[[ ! "${MSG}" =~ ^[0-9-]*$ ]] && continue
+	(( BLOCKS++ ))
+done
+if [ "${BLOCKS}" != "" ]; then
+	echo -e "${ORANGE}${BLOCKS} user(s) are blocked:${NC}${GREY}"
+	sort -r "${BLOCKEDFILE}.jssh"
+	echo -e "${NC}\c"
+else
+	echo -e "${GREEN}No user is blocked currently ...${NC}"
+fi
+# show user created bot stats
+_exec_if_function my_bashbot_stats "$@"
+

bin/delete_message.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/delete_message.sh
+# 
+USAGE='delete_message.sh [-h|--help]  "CHAT[ID]" "MESSAGE[ID]" [debug]'
+# 
+#   DESCRIPTION: delete a message in the given user/group
+# 
+#       OPTIONS: CHAT[ID] - ID number of CHAT or BOTADMIN 
+#                MESSAGE[ID] - message to delete
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 03.01.2021 15:37
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+DELETE="delete_message"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "${3:-debug}" # $3 debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+
+# delete message
+"${DELETE}" "${CHAT}" "$2"
+
+[ "${BOTSENT[OK]}" = "true" ] && BOTSENT[ID]="$2"
+
+# output send message result
+print_result

bin/edit_buttons.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/edit_buttons.sh
+# 
+USAGE='send_message.sh [-h|--help] "CHAT[ID]" "MESSAGE[ID]" "text|url" ...'
+# 
+#   DESCRIPTION: send a send buttons in a row to the given user/group
+# 
+#       OPTIONS: CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                MESSAGE[ID] - ID of MESSAGE with buttons to edit
+#                text|url - buttons to send,  each button as "text|url" pair or
+#                        "url" only to show url as text also, "" starts new row
+#                        "url" not http(s):// or tg:// is sent as callback_data
+#
+#                -h - display short help
+#                --help -  this help
+#
+#       EXAMPLE: 2 buttons on 2 rows, first shows Amazon, second the url as text
+#                send_buttons.sh "Amazon|https://www.amazon.com" "" "https://mydealz.de" ...
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 21.01.2021 08:10
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+SEND="edit_inline_keyboard"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "debug"
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+MESSAGE_ID="$2"
+shift 2
+
+# send message in selected format
+"${SEND}" "${CHAT}" "${MESSAGE_ID}" "$(_button_row "$@")"
+
+# output send message result
+print_result

bin/edit_message.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/edit_message.sh
+# 
+USAGE='send_edit_message.sh [-h|--help] [format|caption] "CHAT[ID]" "MESSAGE[ID]" "message ...." [debug]'
+# 
+#   DESCRIPTION: replace a message in the given user/group
+# 
+#       OPTIONS: format - normal, markdown, html or caption for file caption (optional)
+#                CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                MESSAGE[ID] - message to replace
+#                message - message to send in specified format
+#                    if no format is given send_normal_message() format is used
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 23.12.2020 16:52
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+SEND="edit_normal_message"
+case "$1" in
+	"nor"*|"tex"*)
+		SEND="edit_normal_message"
+		shift
+		;;
+	"mark"*)
+		SEND="edit_markdownv2_message"
+		shift
+		;;
+	"htm"*)
+		SEND="edit_html_message"
+		shift
+		;;
+	"cap"*)
+		SEND="edit_message_caption"
+		shift
+		;;
+esac
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "${4:-debug}" # $4 debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+
+# send message in selected format
+"${SEND}" "${CHAT}" "$2" "$3"
+
+# output send message result
+print_result

bin/kickban_user.sh

@@ -0,0 +1,47 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/kickban_user.sh
+# 
+USAGE='kickban_user.sh [-h|--help] [-u|--unban] "CHAT[ID]" "USER[ID]" [debug]'
+# 
+#   DESCRIPTION: kickban or unban a user from  the given group
+# 
+#       OPTIONS: -u | --unban - unban user
+#                CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                USER[ID] - user to (un)ban
+#
+#                -h - display short help
+#                --help -  this help
+#
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 25.01.2021 20:34
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+BAN="kick_chat_member"
+case "$1" in
+	"-u"|"--unban")
+		BAN="unban_chat_member"
+		shift
+		;;
+esac
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "${3:-debug}" # $3 debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+
+# send message in selected format
+"${BAN}" "$1" "$2"
+
+# output send message result
+print_result

bin/process_batch.sh

@@ -0,0 +1,98 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034,SC2059
+#===============================================================================
+#
+#          FILE: bin/process_batch.sh
+#
+USAGE='process_batch.sh [-h|--help] [-s|--startbot] [-w|--watch] [-n|--lines n] [file] [debug]'
+# 
+#   DESCRIPTION: processes last 10 telegram updates in file, one update per line
+#                 
+#                -s --startbot load addons, start TIMER, trigger startup actions
+#                -w --watch watch for new updates added to file
+#                -n --lines read only last "n" lines
+#                file   to read updates from 
+#			empty means read from webhook pipe
+#                
+#                -h - display short help
+#                --help -  this help
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 27.02.2021 13:14
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+COMMAND="process_multi_updates"
+lines="-n 10"
+mode="batch"
+
+opt=0
+while [[ "${opt}" -lt 5 && "$1" == "-"* ]]
+do
+    (( opt++ )) 
+    case "$1" in
+	"-s"|"--startbot")
+		startbot="yes"
+		shift
+		;;
+	"-w"|"--watch")
+		follow="-f"
+		mode="webhook"
+		shift
+		;;
+	"-n"|"--lines")
+		lines="-n $2"
+		shift 2
+		;;
+    esac
+done
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "debug" # debug
+print_help "${1:-nix}"
+
+# empty file is webhook
+file="${WEBHOOK}"
+[ -n "$1" ] && file="$1"
+
+# start bot
+if [ -n "${startbot}" ]; then
+	# warn when starting bot without pipe
+	[ -p "${file}" ] || printf "%(%c)T: %b\n" -1 "${ORANGE}Warning${NC}: File is not a pipe:${GREY} ${file##*/}${NC}"
+	start_bot "$2" "${mode}"
+	printf "%(%c)T: %b\n" -1 "${GREEN}Bot startup actions done, start ${mode} updates ...${NC}"
+fi
+# check file exist
+if [[ ! -r "${file}" || -d "${file}" ]]; then
+	printf "%(%c)T: %b\n" -1 "${RED}Error${NC}: File not readable:${GREY} ${file}${NC}."
+	exit 1
+fi
+
+####
+# ready, do stuff here -----
+
+# kill all sub processes on exit
+trap 'printf "%(%c)T: %s\n" -1 "Bot in '"${mode}"' mode stopped"; kill $(jobs -p) 2>/dev/null; wait $(jobs -p) 2>/dev/null; send_normal_message "'"${BOTADMIN}"'" "Bot '"${BOTNAME} ${mode}"' stopped ..."' EXIT HUP QUIT
+
+# wait after (first) update to avoid processing to many in parallel
+UPDWAIT="0.5"
+# use tail to read appended updates
+# shellcheck disable=SC2086,SC2248
+tail ${follow} ${lines} "${file}" 2>/dev/null |\
+    while IFS="" read -r input 2>/dev/null
+    do 
+	# read json from stdin and convert update format
+	# replace any ID named BOTADMIN with ID of bot admin
+	: "${input//\"id\":BOTADMIN,/\"id\":${BOTADMIN},}"
+	json='{"result": ['"${_}"']}'
+	UPDATE="$(${JSONSHFILE} -b -n <<<"${json}" 2>/dev/null)"
+
+	# process telegram update
+	"${COMMAND}" "$2"
+	sleep "${UPDWAIT}"
+	UPDWAIT="0.05"
+    done 

bin/process_update.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034,SC2059
+#===============================================================================
+#
+#          FILE: bin/process_update.sh
+#
+USAGE='process_update.sh [-h|--help] [debug] [<file]'
+# 
+#   DESCRIPTION: processes ONE telegram update read from stdin, e.g. form file or webhook
+#                 
+#                -h - display short help
+#                --help -  this help
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 30.01.2021 19:14
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+COMMAND="process_multi_updates"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "debug" # debug
+print_help "${1:-nix}"
+
+
+####
+# ready, do stuff here -----
+
+# read json from stdin and convert update format
+# replace any ID named BOTADMIN with ID of bot admin
+json='{"result": ['"$(cat)"']}'
+json="${json//\"id\":BOTADMIN,/\"id\":${BOTADMIN},}"
+UPDATE="$(${JSONSHFILE} -b -n <<<"${json}" 2>/dev/null)"
+
+# process telegram update
+"${COMMAND}" "$1" 
+

bin/promote_user.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/promote_user.sh
+# 
+USAGE='promote_user.sh [-h|--help] "CHAT[ID]" "USER[ID]" "right[:true|false]" ..'
+# 
+#   DESCRIPTION: promote / denote user rights in given group
+# 
+#       OPTIONS: CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                USER[ID] - user to (un)ban
+#                rights[:true|false] - rights to grant in long or short form,
+#                        followed by :true to grant or :false to renove
+#                  long: is_anonymous can_change_info can_post_messages can_edit_messages
+#                        can_delete_messages can_invite_users can_restrict_members
+#                        can_pin_messages can_promote_member`
+#                  short: anon change post edit delete invite restrict pin promote
+#
+#                -h - display short help
+#                --help - this help
+#
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 25.01.2021 22:34
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+PROMOTE="promote_chat_member"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "debug" # debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+
+# send message in selected format
+"${PROMOTE}" "$@"
+
+# output send message result
+print_result

bin/send_broadcast.sh

@@ -0,0 +1,127 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+# shellcheck disable=SC2059
+#
+#          FILE: bin/broadcast_message.sh
+# 
+USAGE='broadcast_message.sh [-h|--help] [--doit] [--groups|--both|--db=file] [format] "message ...." [debug]'
+# 
+#   DESCRIPTION: send a message to all users listed in a jsonDB (default count db)
+# 
+#       OPTIONS: --doit - broadcast is dangerous, simulate run without --doit
+#                --groups - send to groups instead of users
+#                --both - send to users and groups (default with --db)
+#                --db name - send to all user/groups in jsonDB database (e.g. blocked)
+#                     db file: name.jssh, db keys are user/chat id, values are ignored
+#
+#                format - normal, markdown, html (optional)
+#                message - message to send in specified format
+#                    if no format is givern send_message() format is used
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 16.12.2020 16:14
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# minimum messages seen in a chat before send a broadcast to it
+MINCOUNT=2
+USERDB=""
+
+####
+# broadcast is dangerous, without --doit we do a dry run ...
+if [ "$1" = "--doit" ]; then
+	DOIT="yes"
+	shift
+fi
+
+####
+# send to users by default, --group sends groups, --both to both
+SENDTO="users"
+if [ "$1" = "--both" ]; then
+	GROUPSALSO=" and groups"
+	shift
+elif [ "$1" = "--groups" ]; then
+	SENDTO="groups"
+	GROUPSALSO=" only"
+	shift
+elif [ "$1" = "--db" ]; then
+	USERDB="${2%.jssh}"
+	MINCOUNT=""
+	GROUPSALSO=" and groups"
+	shift 2
+fi
+
+####
+# parse args -----------------
+SEND="send_message"
+case "$1" in
+	"nor"*|"tex"*)
+		SEND="send_normal_message"
+		shift
+		;;
+	"mark"*)
+		SEND="send_markdownv2_message"
+		shift
+		;;
+	"html")
+		SEND="send_html_message"
+		shift
+		;;
+esac
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "$2" # $3 debug
+print_help "$1"
+
+# read in users from given DB or count.jssh 
+database="${USERDB:-${COUNTFILE}}"
+declare -A SENDALL
+jssh_readDB_async "SENDALL" "${database}"
+if [ -z "${SENDALL[*]}" ]; then
+	printf "${ORANGE}User database not found or empty: ${NC}${database}\n"
+fi
+
+	# loop over users
+	printf "${GREEN}Sending broadcast message to ${SENDTO}${GROUPSALSO} of ${BOTNAME} using database:${NC}${GREY} ${database##*/}"
+
+{ 	# dry run
+	[ -z "${DOIT}" ] && printf "${NC}\n${ORANGE}DRY RUN! use --doit as first argument to execute broadcast...${NC}\n"
+
+	for USER in ${!SENDALL[*]}
+	do
+		# send to users, groups or both ...
+		[[ -z "${GROUPSALSO}" && "${USER}" == *"-"* ]] && continue
+		[[ "${SENDTO}" != "users" && "${USER}" != *"-"* ]] && continue
+		# ignore everything not a user or group
+		[[ ! "${USER}" =~ ^[0-9-]*$ ]] && continue
+		# ignore chats with no count or lower MINCOUNT
+		[[ -n "${MINCOUNT}" && ( ! "${SENDALL[${USER}]}" =~ ^[0-9]*$ || "${SENDALL[${USER}]}" -lt "${MINCOUNT}" ) ]] && continue 
+		(( COUNT++ ))
+		if [ -z "${DOIT}" ]; then
+			printf  "${SEND} ${USER} $1 $2\n" 
+		else
+			"${SEND}" "${USER}" "$1" "$2"
+			printf  "." 1>&2
+			# ups, kicked or banned ...
+			if [ "${BOTSENT[ERROR]}" = "403" ]; then
+				# remove chat from future broadcasts
+				jssh_insertKeyDB "${USER}" "${SENDALL[${USER}]} banned" "${COUNTFILE}"
+				printf "${ORANGE}Warning: bot banned from chat${NC} %s ${ORANGE}after${NC} %s ${ORANGE}commands${NC}\n"\
+						"${USER}" "${SENDALL[${USER}]}"
+			fi
+			sleep 0.1
+		fi
+	done
+	# printout final stats message
+	printf "${NC}\n${GREEN}Message${NC} $1 ${GREEN}sent to${NC} ${COUNT} ${GREEN}${SENDTO}${GROUPSALSO}.${NC}\n"
+} | more
+

bin/send_buttons.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/send_message.sh
+# 
+USAGE='send_message.sh [-h|--help] "CHAT[ID]" "message" "text|url" ...'
+# 
+#   DESCRIPTION: send a send buttons in a row to the given user/group
+# 
+#       OPTIONS: CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                message - message to send
+#                text|url - buttons to send,  each button as "text|url" pair or
+#                        "url" not http(s):// or tg:// is sent as callback_data
+#                        "url" only to show url as text also, "" starts new row
+#
+#                -h - display short help
+#                --help -  this help
+#
+#       EXAMPLE: 2 buttons on 2 rows, first shows Amazon, second the url as text
+#                send_buttons.sh "Amazon|https://www.amazon.com" "" "https://mydealz.de" ...
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 18.01.2021 11:34
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+SEND="send_inline_keyboard"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "debug"
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+MESSAGE="$2"
+shift 2
+
+# send message in selected format
+"${SEND}" "${CHAT}" "${MESSAGE}" "$(_button_row "$@")"
+
+# output send message result
+print_result

bin/send_dice.sh

@@ -0,0 +1,47 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/send_dice.sh
+#
+USAGE='send_dice.sh [-h|--help] "CHAT[ID]" "emoji" [debug]'
+# 
+#   DESCRIPTION: send an animated emoji (dice) to given chat
+# 
+#       OPTIONS: CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                emoji - must be one of: “🎲”, “🎯”, “🏀”, “⚽” “🎰” "🎳"
+#                        :game_die: :dart: :basketball: :soccer: :slot_machine: :bowling:
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 07.02.2021 18:45
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+SEND="send_dice"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "${3:-debug}" # $5 debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+
+# send message in selected format
+"${SEND}" "${CHAT}" "$2"
+
+# output send message result
+print_result

bin/send_file.sh

@@ -0,0 +1,55 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/send_file.sh
+#
+USAGE='send_file.sh [-h|--help] "CHAT[ID]" "file|URL" "caption ...." [type] [debug]'
+# 
+#   DESCRIPTION: send a file to the given user/group
+# 
+#       OPTIONS: CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                file - local file to send, must be an absolute path or relative to pwd
+#                       Note: must not contain .. or . and located below BASHBOT_ETC
+#                URL - send an URL instead local file
+#
+#                caption - message to send with file
+#                type - photo, video, sticker, voice, document (optional)
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 25.12.2020 20:24
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+SEND="send_file"
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "${5:-debug}" # $5 debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+
+FILE="$2"
+# convert to absolute path if not start with / or http://
+[[ ! ( "$2" == "/"* ||  "$2" =~ ^https*:// || "$2" == "file_id://"*) ]] && FILE="${PWD}/$2"
+
+# send message in selected format
+"${SEND}" "${CHAT}" "${FILE}" "$3" "$4"
+
+# output send message result
+print_result

bin/send_message.sh

@@ -0,0 +1,73 @@
+#!/bin/bash
+# shellcheck disable=SC1090,SC2034
+#===============================================================================
+#
+#          FILE: bin/send_message.sh
+# 
+USAGE='send_message.sh [-h|--help] [format] "CHAT[ID]" "message ...." [debug]'
+# 
+#   DESCRIPTION: send a message to the given user/group
+# 
+#       OPTIONS: format - normal, markdown, html, stdin, - (optional)
+#                CHAT[ID] - ID number of CHAT or BOTADMIN to send to yourself
+#                message - message to send in specified format
+#                    if no format is givern send_message() format is used
+#
+#                use format "stdin" to read message from stdin or from a file:
+#                  send_message.sh stdin "CHAT[ID]" <file
+#                  df -h | send_message.sh - "CHAT[ID]"
+#
+#                -h - display short help
+#                --help -  this help
+#
+#                Set BASHBOT_HOME to your installation directory
+#
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#       CREATED: 16.12.2020 11:34
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
+
+####
+# parse args
+SEND="send_message"
+case "$1" in
+	"nor"*|"tex"*)
+		SEND="send_normal_message"
+		shift
+		;;
+	"mark"*)
+		SEND="send_markdownv2_message"
+		shift
+		;;
+	"html")
+		SEND="send_html_message"
+		shift
+		;;
+	"stdin"|"-")
+		FILE="stdin"
+		shift
+		;;
+esac
+
+# set bashbot environment
+source "${0%/*}/bashbot_env.inc.sh" "${3:-debug}" # $3 debug
+print_help "$1"
+
+####
+# ready, do stuff here -----
+if [ "$1" == "BOTADMIN" ]; then
+	CHAT="${BOTADMIN}"
+else
+	CHAT="$1"
+fi
+
+# send message in selected format
+if [ "${FILE}" = "stdin" ]; then
+	"${SEND}" "${CHAT}" "$(cat)"
+else
+	"${SEND}" "${CHAT}" "$2"
+fi
+# output send message result
+print_result

commands.sh

@@ -8,18 +8,17 @@
 # | |__/ / |_| |  | | | | |_| | |__   | |____( (_| | | |__ _ 
 # |_____/ \___/   |_| |_|\___/ \___)  |_______)____|_|\___)_|
 #
-# this file *MUST* not be edited! place your config and commands in
-# the file "mycommands.sh". a clean version is provided as "mycommands.sh.clean"
+# this file *MUST* not edited! place your config in the file "mycommands.conf"
+# and commands in "mycommands.sh", a clean version is provided as "mycommands.sh.clean"
 #
 
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 #
 
-# adjust your language setting here, e.g. when run from other user or cron.
-# https://github.com/topkecleon/telegram-bot-bash#setting-up-your-environment
+# bashbot locale defaults to c.UTF-8, adjust locale in mycommands.sh if needed
 export 'LC_ALL=C.UTF-8'
 export 'LANG=C.UTF-8'
 export 'LANGUAGE=C.UTF-8'
@@ -38,27 +37,30 @@ It currently can send, receive and forward messages, custom keyboards, photos, a
 #-----------------------------
 # this file *MUST* not edited!
 # copy "mycommands.sh.dist" to "mycommands.sh" and change the strings there
-bashbot_help='Place your own commands and messages in mycommands.sh
-
+bashbot_help='
 *Available commands*:
 *• /start*: _Start bot and get this message_.
 *• /help*: _Get this message_.
 *• /info*: _Get shorter info message about this bot_.
-*• /question*: _Start interactive chat_.
-*• /cancel*: _Cancel any currently running interactive chats_.
-*• /kickme*: _You will be autokicked from the chat_.
+*• /kickme*: _You will be autokicked from the group_.
 *• /leavechat*: _The bot will leave the group with this command _.
-Written by Drew (@topkecleon), Daniil Gentili (@danogentili) and KayM(@gnadelwartz).
-Get the code in my [GitHub](http://github.com/topkecleon/telegram-bot-bash)
+Additional commands from mycommands.dist ...
+*• /game*: _throw a die_.
+*• /question*: _Start interactive chat_.
+*• /cancel*: _Cancel any currently running interactive chat_.
+*• /run_notify*: _Start background job_.
+*• /stop_notify*: _Stop notify background job_.
+Written by Drew (@topkecleon) and KayM (@gnadelwartz).
+Get the code on [GitHub](http://github.com/topkecleon/telegram-bot-bash)
 '
 
 # load modules on startup and always on on debug
-if [ -n "${1}" ]; then
+if [ -n "$1" ]; then
     # load all readable modules
     for modules in "${MODULEDIR:-.}"/*.sh ; do
-	if [[ "${1}" == *"debug"* ]] || ! _is_function "$(basename "${modules}")"; then
+	if [[ "$1" == *"debug"* ]] || ! _is_function "$(basename "${modules}")"; then
 		# shellcheck source=./modules/aliases.sh
-		[ -r "${modules}" ] && source "${modules}" "${1}"
+		[ -r "${modules}" ] && source "${modules}" "$1"
 	fi
     done
 fi
@@ -66,30 +68,44 @@ fi
 #----------------------------
 # this file *MUST* not edited!
 # copy "mycommands.sh.dist" to "mycommands.sh" and change the values there
-# defaults to no inline and nonsense home dir
+# defaults to no inline, all commands  and nonsense home dir
 export INLINE="0"
+export CALLBACK="0"
+export MEONLY="0"
 export FILE_REGEX="${BASHBOT_ETC}/.*"
 
 
 # load mycommands
 # shellcheck source=./commands.sh
-[ -r "${BASHBOT_ETC:-.}/mycommands.sh" ] && source "${BASHBOT_ETC:-.}/mycommands.sh"  "${1}"
+[ -r "${BASHBOT_ETC:-.}/mycommands.sh" ] && source "${BASHBOT_ETC:-.}/mycommands.sh"  "$1"
 
 
-if [ -z "${1}" ] || [[ "${1}" == *"debug"* ]];then
-    # detect inline commands....
-    # no default commands, all processing is done in myinlines()
-    if [ "$INLINE" != "0" ] && [ -n "${iQUERY[ID]}" ]; then
-    	# forward iinline query to optional dispatcher
-	_exec_if_function myinlines
+if [ -z "$1" ] || [[ "$1" == *"debug"* ]];then
+    #################
+    # detect inline and callback query
+    if [ -n "${iQUERY[ID]}" ]; then
+    	# forward inline query to optional dispatcher
+	[ "${INLINE:-0}" != "0" ] &&  _exec_if_function myinlines
 
-    # regular (global) commands ...
-    # your commands are in mycommands() 
+    elif [ -n "${iBUTTON[ID]}" ]; then
+    	# forward inline query to optional dispatcher
+	[ "${CALLBACK:-0}" != "0" ] && _exec_if_function mycallbacks
+
+    #################
+    # regular command
     else
+	
+	###################
+	# if is bashbot is group admin it get commands sent to other bots
+	# set MEONLY=1 to ignore commands for other bots
+	if [[ "${MEONLY}" != "0" && "${MESSAGE}" == "/"*"@"* ]]; then
+		# here we have a command with @xyz_bot added, check if it's our bot
+		[ "${MESSAGE%%@*}" != "${MESSAGE%%@${ME}}" ] && return
+	fi 
 
 	###################
 	# user defined commands must placed in mycommands
-	_exec_if_function mycommands
+	! _is_function mycommands || mycommands
 
 	# run commands if true (0) is returned or if mycommands dose not exist
 	# shellcheck disable=SC2181
@@ -105,7 +121,10 @@ if [ -z "${1}" ] || [[ "${1}" == *"debug"* ]];then
 			;;
 		'/start'*)
 			send_action "${CHAT[ID]}" "typing"
-			user_is_botadmin "${USER[ID]}" && send_markdown_message "${CHAT[ID]}" "You are *BOTADMIN*."
+			MYCOMMANDS="*Note*: MISSING mycommands.sh:  copy _mycommands.dist_ or _mycommands.clean_."
+			[ -r "${BASHBOT_ETC:-.}/mycommands.sh" ] && MYCOMMANDS="Place your commands and messages in _mycommands.sh_"
+			user_is_botadmin "${USER[ID]}" &&\
+				send_markdownv2_message "${CHAT[ID]}" "You are *BOTADMIN*.\n${MYCOMMANDS}"
 			if user_is_admin "${CHAT[ID]}" "${USER[ID]}" || user_is_allowed  "${USER[ID]}" "start" ; then
 				send_markdown_message "${CHAT[ID]}" "${bashbot_help}"
 			else
@@ -116,7 +135,7 @@ if [ -z "${1}" ] || [[ "${1}" == *"debug"* ]];then
 		'/help'*)
 			send_markdown_message "${CHAT[ID]}" "${bashbot_help}"
 			;;
-		'/leavechat'*) # bot leave chat if user is admin in chat
+		'/leavechat'*)	# bot leave chat if user is admin in chat
 			if user_is_admin "${CHAT[ID]}" "${USER[ID]}" || user_is_allowed  "${USER[ID]}" "leave" ; then
 				send_markdown_message "${CHAT[ID]}" "*LEAVING CHAT...*"
    				leave_chat "${CHAT[ID]}"

db.json

@@ -2,8 +2,8 @@
   "usage": 
      {
 	"ok":true,
-	"url":"https://my-json-server.typicode.com/topkecleon/telegram-bot-bash/",
-	"description":"Testing of JSON responses for github.com/topkecleon/telegram-bot-bash"
+	"url":"https://my-json-server.typicode.com/topkecleon/telegram-bot-bash/usage?",
+	"description":"Testing of JSON responses for github.com/topkecleon/telegram-bot-bash, replace usage with the dataset you want to query"
     } ,
   "false": 
      {
@@ -11,7 +11,7 @@
 	"error_code":404,
 	"description":"Not Found"
     } ,
-  "ok": 
+  "true": 
     {
 	"ok":true,
 	"description":"Test for ok"
@@ -27,5 +27,4 @@
 		"username":"TestBotBash"
 	}
     }
-  
 }

dev/all-tests.sh

@@ -1,61 +1,79 @@
 #!/usr/bin/env bash
-# this has to run once atfer git clone
-# and every time we create new hooks
-#### $$VERSION$$ v1.1-0-gc0eb399
+#############################################################
+#
+# File: dev/all-tests.sh
+#
+# Description: run all tests, exit after failed test
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#############################################################
 
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
-if [ "$GIT_DIR" != "" ] ; then
-	cd "$GIT_DIR/.." || exit 1
-else
-	echo "Sorry, no git repository $(pwd)" && exit 1
-fi
+#shellcheck disable=SC1090
+source "${0%/*}/dev.inc.sh"
 
+##########################
 # create test environment
 TESTENV="/tmp/bashbot.test$$"
 mkdir "${TESTENV}"
 cp -r ./* "${TESTENV}"
 cd "test" || exit 1
 
+# delete possible config
+rm -f "${TESTENV}/botconfig.jssh" "${TESTENV}/botacl" 2>/dev/null
+
+# mkdir needed dirs
+mkdir "${TESTENV}/data-bot-bash"
+
+# inject JSON.sh
+mkdir "${TESTENV}/JSON.sh"
+curl -sL "https://cdn.jsdelivr.net/gh/dominictarr/JSON.sh/JSON.sh" >"${TESTENV}/JSON.sh/JSON.sh"
+chmod +x "${TESTENV}/JSON.sh/JSON.sh"
+
+########################
+#prepare and run tests
 #set -e
 fail=0
 tests=0
 passed=0
 #all_tests=${__dirname:}
-#echo PLAN ${#all_tests}
+#printf PLAN ${#all_tests}
 for test in $(find ./*-test.sh | sort -u) ;
 do
   [ "${test}" = "dev/all-tests.sh" ] && continue
   [ ! -x "${test}" ] && continue
   tests=$((tests+1))
-  echo "TEST: ${test}"
+  printf "TEST: %s\n" "${test}"
   "${test}" "${TESTENV}"
   ret=$?
-  if [ "$ret" -eq 0 ] ; then
-    echo "OK: ---- ${test}"
+  set +e
+  if [ "${ret}" -eq 0 ] ; then
+    printf "OK: ---- %s\n" "${test}"
     passed=$((passed+1))
   else
-    echo "FAIL: $test $fail"
+    printf "FAIL: %s\n" "${test} ${fail}"
     fail=$((fail+ret))
     break
   fi
 done
 
-if [ "$fail" -eq 0 ]; then
-  /bin/echo -n 'SUCCESS '
+###########################
+# cleanup depending on test state
+if [ "${fail}" -eq 0 ]; then
+  printf 'SUCCESS '
   exitcode=0
   rm -rf "${TESTENV}"
 else
-  /bin/echo -n 'FAILURE '
+  printf 'FAILURE '
   exitcode=1
   rm -rf "${TESTENV}/test"
   find "${TESTENV}/"* ! -name '[a-z]-*' -delete
 fi
 
-echo -e "${passed} / ${tests}\\n"
-[ -d "${TESTENV}" ] && echo "Logfiles from run are in ${TESTENV}"
+#########################
+# show test result and test logs
+printf "%s\n\n" "${passed} / ${tests}"
+[ -d "${TESTENV}" ] && printf "Logfiles from run are in %s\n" "${TESTENV}"
 
-ls -ld /tmp/bashbot.test* 2>/dev/null && echo "Do not forget to delete bashbot test files in /tmp!!"
+ls -ld /tmp/bashbot.test* 2>/dev/null && printf "Do not forget to delete bashbot test files in /tmp!!\n"
 
-exit ${exitcode}
+exit "${exitcode}"

dev/dev.inc.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+#############################################################
+#
+# File: dev/dev.inc.sh
+#
+# Description: common stuff for all dev scripts
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#############################################################
+
+# magic to ensure that we're always inside the root of our application,
+# no matter from which directory we'll run script
+
+# shellcheck disable=SC2034
+GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
+BASE_DIR=$(git rev-parse --show-toplevel 2>/dev/null)
+if [ "${BASE_DIR}" != "" ] ; then
+	cd "${BASE_DIR}" || exit 1
+else
+	printf "Sorry, no git repository %s\n" "$(pwd)" && exit 1
+fi
+
+HOOKDIR="dev/hooks"
+LASTCOMMIT=".git/.lastcommit"

dev/git-add.sh

@@ -3,32 +3,37 @@
 #
 # works together with git pre-push.sh and ADD all changed files since last push
 
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
-if [ "$GIT_DIR" != "" ] ; then
-	cd "$GIT_DIR/.." || exit 1
-else
-	echo "Sorry, no git repository $(pwd)" && exit 1
+#shellcheck disable=SC1090
+source "${0%/*}/dev.inc.sh"
+
+# check for last commit date
+if [ ! -f "${LASTCOMMIT}" ]; then
+	if ! touch -d "$(git log -1 --format=%cD)" "${LASTCOMMIT}"; then
+		printf "No previous commit found, use \"git add\" instead ... Abort\n"
+		exit
+	fi
 fi
 
-[ ! -f .git/.lastpush ] && echo "No push or hooks not installed, use \"git add\" instead ... Abort" && exit
-
-FILES="$(find ./* -newer .git/.lastpush)"
-[ "${FILES}" = "" ] && echo "Noting changed since last push ... Abort" && exit
+set +f
+FILES="$(find ./*  -newer "${LASTCOMMIT}" | grep -v -e 'DIST\/' -e 'STANDALONE\/' -e 'JSON.sh')"
+set -f
+# FILES="$(find ./* -newer .git/.lastpush)"
+[ "${FILES}" = "" ] && printf "Nothing changed since last commit ...\n" && exit
 
 # run pre_commit on files
 dev/hooks/pre-commit.sh
 
-echo -n "Add files to repo: "
+printf "Add files to repo: "
 # shellcheck disable=SC2086
 for file in ${FILES}
 do
 	[ -d "${file}" ] && continue
-	echo -n "${file} "
+	printf "%s" "${file} "
 done
-git add .
-echo "done."
+printf " - Done.\n"
+
+# stay with "." for (re)moved files!
+git add .
 

dev/hooks/post-commit.sh

@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+#### $$VERSION$$ v1.52-1-g0dae2db
+
+############
+# NOTE: you MUST run install-hooks.sh again when updating this file!
+
+# magic to ensure that we're always inside the root of our application,
+# no matter from which directory we'll run script
+GIT_DIR=$(git rev-parse --git-dir)
+cd "${GIT_DIR}/.." || exit 1
+
+export HOOKDIR="dev/hooks"
+LASTPUSH='.git/.lastcommit'
+
+# if any command inside script returns error, exit and return that error 
+set -e
+
+#printf "Running post-commit hook\n............................\n"
+
+unset IFS; set -f
+
+# note date of last push for version
+touch "${LASTPUSH}"

dev/hooks/pre-commit.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 ############
 # NOTE: you MUST run install-hooks.sh again when updating this file!
@@ -7,7 +7,7 @@
 # magic to ensure that we're always inside the root of our application,
 # no matter from which directory we'll run script
 GIT_DIR=$(git rev-parse --git-dir)
-cd "$GIT_DIR/.." || exit 1
+cd "${GIT_DIR}/.." || exit 1
 
 export HOOKDIR="dev/hooks"
 LASTPUSH='.git/.lastpush'
@@ -15,64 +15,60 @@ LASTPUSH='.git/.lastpush'
 # if any command inside script returns error, exit and return that error 
 set -e
 
-echo "Running pre-commit hook"
-echo "............................" 
+printf "Running pre-commit hook\n............................\n" 
 
 unset IFS; set -f
 
 # check for shellcheck
 if command -v  shellcheck >/dev/null 2>&1; then
-	echo "Test all scripts with shellcheck"
+	printf "Test all scripts with shellcheck\n"
 else
-	echo "Error: shellcheck is not installed. Please install shellcheck"
+	printf "Error: shellcheck is not installed. Please install shellcheck\n"
 	exit 1
 fi
 
 # run shellcheck before commit
 set +f
-FILES="$(find ./* -name '*.sh' | grep -v 'DIST\/' | grep -v 'STANDALONE\/')"
+FILES="$(find ./* -name '*.sh' | grep -v -e 'DIST\/' -e 'STANDALONE\/' -e 'JSON.sh')"
 set -f
 FILES="${FILES} $(sed '/^#/d' <"dev/shellcheck.files")"
-if [ "$FILES" != "" ]; then
+if [ "${FILES}" != "" ]; then
 	# shellcheck disable=SC2086
-	shellcheck -x ${FILES} || exit 1
-	echo "    OK"
-	echo "............................" 
+	shellcheck -o all -e SC2249,SC2154 -x ${FILES} || exit 1
+	printf "    OK\n............................\n"
 else
 	# something went wrong
 	exit 1
 fi
 
+# get version strings
 REMOTEVER="$(git ls-remote -t --refs 2>/dev/null | tail -1 | sed -e 's/.*\/v//' -e 's/-.*//')"
 VERSION="$(git describe --tags | sed -e 's/-.*//' -e 's/v//' -e 's/,/./')"
-
+[ -z "${REMOTEVER}" ] && REMOTEVER="${VERSION}"
 
 # LOCAL version must greater than latest REMOTE release version
-echo "Update Version of modified files" 
-if (( $(echo "${VERSION} >= ${REMOTEVER}" | bc -l) )); then
+printf "Update Version of modified files\n"
+if ! command -v bc &> /dev/null || (( $(printf "%s\n" "${VERSION} >= ${REMOTEVER}" | bc -l) )); then
 	# update version in bashbot files on push
 	set +f
-	[ -f "${LASTPUSH}" ] && LASTFILES="$(find ./* -newer "${LASTPUSH}")"
+	[ -f "${LASTPUSH}" ] && LASTFILES="$(find ./* -newer "${LASTPUSH}" ! -path "./DIST/*" ! -path "./STANDALONE/*")"
 	[ "${LASTFILES}" = "" ] && exit
-	echo -n " "
+	printf " "
 	# shellcheck disable=SC2086
 	dev/version.sh ${LASTFILES} 2>/dev/null || exit 1
-	echo "    OK"
-	echo "............................" 
+	printf "    OK\n............................\n"
 else
-	echo "Error: local version ${VERSION} must be equal to or greater then release version ${REMOTEVER}."
-        echo "use \"git tag vx.zz\" to create a new local version"
+	printf "Error: local version %s must be equal to or greater then release version%s\n" "${VERSION}" "${REMOTEVER}."
+        printf "use \"git tag vx.zz\" to create a new local version\n"
 	exit 1
 fi
 
 if command -v codespell &>/dev/null; then
-	echo "Running codespell"
-	echo "............................" 
-	codespell -B 1 --skip="*.log,*.html,*.txt,.git*,jsonDB-keyboard" -L "ba"
-	echo "if there are (to many) typo's shown, consider running:"
-	echo "codespell -i 3 -w --skip=\"*.log,*.html,*.txt,.git*,examples\" -L \"ba\""
+	printf "Running codespell\n............................\n"
+	codespell -q 3 --skip="*.zip,*gz,*.log,*.html,*.txt,.git*,jsonDB-keyboard,DIST,STANDALONE" -L "ba"
+	printf "if there are (to many) typo's shown, consider running:\ncodespell -i 3 -w --skip=\"*.log,*.html,*.txt,.git*,examples\" -L \"ba\"\n"
 else
-	echo "consider installing codespell: pip install codespell"
+	printf "consider installing codespell: pip install codespell\n"
 fi
-echo "............................" 
+printf "............................\n" 
 

dev/hooks/pre-push.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 ############
 # NOTE: you MUST run install-hooks.sh again when updating this file!
@@ -7,7 +7,7 @@
 # magic to ensure that we're always inside the root of our application,
 # no matter from which directory we'll run script
 GIT_DIR=$(git rev-parse --git-dir)
-cd "$GIT_DIR/.." || exit 1
+cd "${GIT_DIR}/.." || exit 1
 
 export HOOKDIR="dev/hooks"
 LASTPUSH='.git/.lastpush'
@@ -15,8 +15,7 @@ LASTPUSH='.git/.lastpush'
 # if any command inside script returns error, exit and return that error 
 set -e
 
-echo "Running pre-push hook"
-echo "............................" 
+printf "Running pre-push hook\n............................\n"
 
 unset IFS; set -f
 

dev/inject-json.sh

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+##############################################################
+#
+# File: inject-json.sh
+#
+# Description: download and prepare JSON.sh and JSON.awk
+#
+# Usage: source inject-json.sh
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+##############################################################
+
+# download JSON.sh
+JSONSHFILE="JSON.sh/JSON.sh"
+if [ ! -r "${JSONSHFILE}" ]; then
+	printf "Inject JSON.sh ... "
+	mkdir "JSON.sh" 2>/dev/null
+	curl -sL -o "${JSONSHFILE}" "https://cdn.jsdelivr.net/gh/dominictarr/JSON.sh/JSON.sh"
+	chmod +x "${JSONSHFILE}"
+	printf "Done!\n"
+fi
+
+# download JSON.awk
+JSONSHFILE="JSON.sh/JSON.awk.dist"
+if [ ! -r "${JSONSHFILE}" ]; then
+	printf "Inject JSON.awk ... "
+	curl -sL -o "${JSONSHFILE}" "https://cdn.jsdelivr.net/gh/step-/JSON.awk/JSON.awk" 
+	curl -sL -o "${JSONSHFILE%/*}/awk-patch.sh" "https://cdn.jsdelivr.net/gh/step-/JSON.awk/tool/patch-for-busybox-awk.sh"
+	chmod +x "${JSONSHFILE}"
+	printf "Done!\n"
+	bash "${JSONSHFILE%/*}/awk-patch.sh" "${JSONSHFILE}"
+fi
+# delete backup files
+rm -f "${JSONSHFILE%/*}"/*.bak
+

dev/install-hooks.sh

@@ -1,26 +1,18 @@
 #!/usr/bin/env bash
 # this has to run once atfer git clone
 # and every time we create new hooks
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
-if [ "$GIT_DIR" != "" ] ; then
-	cd "$GIT_DIR/.." || exit 1
-else
-	echo "Sorry, no git repository $(pwd)" && exit 1
-fi
+#shellcheck disable=SC1090
+source "${0%/*}/dev.inc.sh"
 
-HOOKDIR="dev/hooks"
-
-echo -n "Installing hooks..."
-for hook in pre-commit pre-push
+printf "Installing hooks..."
+for hook in pre-commit post-commit pre-push
 do
    rm -f "${GIT_DIR}/hooks/${hook}"
    if [ -f "${HOOKDIR}/${hook}.sh" ]; then
-	echo -n " $hook"
+	printf "%s"" ${hook}"
 	ln -s "../../${HOOKDIR}/${hook}.sh" "${GIT_DIR}/hooks/${hook}"
    fi
 done
-echo " Done!"
+printf " Done!\n"

dev/make-distribution.sh

@@ -1,74 +1,85 @@
 #!/usr/bin/env bash
-# file: make-distribution.sh
-# creates files and arcchives to dirtribute bashbot
+##############################################################
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+# File: make-distribution.sh
+#
+# Description: creates files and arcchives to distribute bashbot
+#
+# Options: --notest - skip tests
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+##############################################################
 
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
-if [ "$GIT_DIR" != "" ] ; then
-	cd "$GIT_DIR/.." || exit 1
-else
-	echo "Sorry, no git repository $(pwd)" && exit 1
-fi
+#shellcheck disable=SC1090
+source "${0%/*}/dev.inc.sh"
 
 VERSION="$(git describe --tags | sed -e 's/-[0-9].*//' -e 's/v//')"
 
 DISTNAME="telegram-bot-bash"
 DISTDIR="./DIST/${DISTNAME}" 
-DISTFILES="bashbot.rc bashbot.sh commands.sh mycommands.sh mycommands.sh.clean doc examples scripts modules addons LICENSE README.md README.txt README.html"
+DISTMKDIR="data-bot-bash logs bin bin/logs addons"
+
+DISTFILES="bashbot.sh commands.sh mycommands.sh.clean bin doc examples scripts modules LICENSE README.md README.txt README.html"
+DISTFILESDEV="dev/make-standalone.sh dev/inject-json.sh dev/make-html.sh dev/obfuscate.sh"
+DISTFILESDIST="mycommands.sh mycommands.conf bashbot.rc $(echo "addons/"*.sh)"
 
 # run tests first!
-
-for test in dev/all-test*.sh
+for test in $1 dev/all-test*.sh
 do
+   [[ "${test}" == "--notest"* ]] && break
    [ ! -x "${test}" ] && continue
    if ! "${test}" ; then
-	echo "Test ${test} failed, can't create dist!"
+	printf "ERROR: Test %s failed, can't create dist!\n" "${test}"
 	exit 1
   fi
 done
 
 # create dir for distribution and copy files
 mkdir -p "${DISTDIR}" 2>/dev/null
-# shellcheck disable=SC2086
-echo "Copy files"
+
+printf "Copy files\n"
 # shellcheck disable=SC2086
 cp -r ${DISTFILES} "${DISTDIR}"
+mkdir -p "${DISTDIR}/dev"
+# shellcheck disable=SC2086
+cp ${DISTFILESDEV} "${DISTDIR}/dev"
 cd "${DISTDIR}" || exit 1
 
-# do not overwrite on update
-echo "Create .dist files"
-for file in mycommands.sh bashbot.rc addons/*.sh
+printf "Create directories\n"
+# shellcheck disable=SC2250
+for dir in $DISTMKDIR
 do
-	[ "${file}" = "addons/*.sh" ] && continue
-	mv "${file}" "${file}.dist"
+	[ ! -d "${dir}" ] && mkdir "${dir}"
 done
 
-# dwonload JSON.sh
-echo "Inject JSON.sh"
-JSONSHFILE="JSON.sh/JSON.sh"
-if [ ! -f "${JSONSHFILE}" ]; then
-	mkdir "JSON.sh" 2>/dev/null
-	curl -sL -o "${JSONSHFILE}" "https://cdn.jsdelivr.net/gh/dominictarr/JSON.sh/JSON.sh"
-	chmod +x "${JSONSHFILE}" 
-fi
+# do not overwrite on update
+printf "Create .dist files\n"
+for file in ${DISTFILESDIST}
+do
+	[ "${file}" = "addons/*.sh" ] && continue
+	cp "${BASE_DIR}/${file}" "${file}.dist"
+done
+
+# inject JSON.sh into distribution
+# shellcheck disable=SC1090
+source "${BASE_DIR}/dev/inject-json.sh"
 
 # make html doc
-echo "Create html doc"
-#shellcheck disable=SC1090
-source "$GIT_DIR/../dev/make-html.sh"
+printf "Create html doc\n"
+# shellcheck disable=SC1090,SC1091
+source "../../dev/make-html.sh"
 
 # create archive
 cd .. || exit 1
-echo "Create dist archives"
-zip -rq "${DISTNAME}-${VERSION}.zip" "${DISTNAME}"
-tar -czf "${DISTNAME}-${VERSION}.tar.gz" "${DISTNAME}"
+printf "Create dist archives\n"
+# shellcheck disable=SC2046
+zip -rq - "${DISTNAME}" --exclude $(cat  "${BASE_DIR}/dev/${0##*/}.exclude") >"${DISTNAME}-${VERSION}.zip"
+tar --exclude-ignore="${BASE_DIR}/dev/${0##*/}.exclude" -czf "${DISTNAME}-${VERSION}.tar.gz" "${DISTNAME}"
 
-echo "Done!"
+printf "%s Done!\n" "$0"
 
 # shellcheck disable=SC2086
-ls -ld ${DISTNAME}-${VERSION}.*
-
+ls -ld "${DISTNAME}-${VERSION}".*
 
+# an empty DEBUG.log is created ... :-(
+rm -f "${BASE_DIR}/test/"*.log

dev/make-distribution.sh.exclude

@@ -0,0 +1,23 @@
+STANDALONE
+data-bot-bash/*
+test
+webhook-fifo*
+JSON.awk
+bashbot.rc
+mycommands.sh
+mycommands.conf
+awk-patch.sh
+make-standalone.sh.include
+*.jssh*
+botacl
+*.flock
+*.log
+*.last
+*.ok
+*.bad
+*.bak
+*.jpg
+*.jpeg
+*.png
+*.zip
+*.gz

dev/make-html.sh

@@ -1,16 +1,44 @@
 #!/usr/bin/env bash
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-if [ ! -f README.html ]; then
-	echo "This script must run where README.html is!" && exit 1
-fi
+##############################################################
+#
+# File: make-html.sh
+#
+# Description: creates html version from *.md files
+#
+# Usage: source make-hmtl
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+##############################################################
 
-# make html doc
-mkdir html  2>/dev/null
-cp README.html html/index.html
-find doc -iname "*.md" -type f -exec sh -c 'pandoc -s -f commonmark -M "title=Bashobot Documentation - ${0%.md}.html"  "${0}" -o "./html/$(basename ${0%.md}.html)"' {} \;
-if [ -d "examples" ]; then
-	find examples -iname "*.md" -type f -exec sh -c 'pandoc -s -f commonmark -M "title=Bashobot Documentation - ${0%.md}.html"  "${0}" -o "${0%.md}.html"' {} \;
-	EXAMPLES="examples"
+# check for correct dir
+if [[ ! ( -f README.html && -f README.md )  ]]; then
+    printf "Error: Can't create html, script must run where README.md and README.html is!\n"
+
+else
+    # check if pandoc installed
+    if [ "$(type -t pandoc)" != "file" ]; then
+	printf "pandoc not found, skipping html generation ...\n"
+
+    else
+	########
+	# everything seems ok, start html generation
+	printf "Start hmtl conversion "
+	# create dir for html doc and index.html there
+	mkdir html 2>/dev/null
+	cp README.html html/index.html
+	# convert *.md files in doc to *.hmtl in html
+	find doc -iname "*.md" -type f -exec sh -c\
+		 'printf "."; pandoc -s -f commonmark -M "title=Bashobot Documentation - ${0%.md}.html"  "$0" -o "./html/$(basename ${0%.md}.html)"' {} \;
+	# html for examples dir
+	if [ -d "examples" ]; then
+		EXAMPLES="examples" # add to final conversion job
+		find examples -iname "*.md" -type f -exec sh -c\
+			'printf "."; pandoc -s -f commonmark -M "title=Bashobot Documentation - ${0%.md}.html"  "$0" -o "${0%.md}.html"' {} \;
+	fi
+	# final: convert links from *.md to *.html
+	# shellcheck disable=SC2248
+	find README.html html ${EXAMPLES} -iname "*.html" -type f -exec sh -c\
+		'sed -i -E "s/href=\"(\.\.\/)*doc\//href=\"\1html\//g;s/href=\"(.*).md(#.*)*\"/href=\"\1.html\"/g" $0' {} \;
+	printf " Done!\n"
+    fi
 fi
-find README.html html ${EXAMPLES} -iname "*.html" -type f -exec sh -c 'sed -i -E "s/href=\"(\.\.\/)*doc\//href=\"\1html\//g;s/href=\"(.*).md(#.*)*\"/href=\"\1.html\"/g" ${0}' {} \;

dev/make-standalone.sh

@@ -1,44 +1,82 @@
 #!/usr/bin/env bash
-# file: make-standalone.sh
-# even after make-distribution.sh bashbot is not self contained as it was in the past.
+###################################################################
 #
-# If you your bot is finished you can use make-standalone.sh to create the
-# the old all-in-one bashbot:  bashbot.sh and commands.sh only!
+# File: make-standalone.sh
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+# Description:
+#    even after make-distribution.sh bashbot is not self contained as it was in the past.
+#
+# Options: --notest
+#
+#   If you your bot is finished you can use make-standalone.sh to create the
+#    the old all-in-one bashbot:  bashbot.sh and commands.sh only!
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+###################################################################
 
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
-if [ "$GIT_DIR" != "" ] ; then
-	cd "$GIT_DIR/.." || exit 1
-else
-	[ ! -f "bashbot.sh" ] && echo "bashbot.sh not found in $(pwd)" && exit 1
+# include git config and change to base dir
+incfile="${0%/*}/dev.inc.sh"
+#shellcheck disable=SC1090
+[ -f "${incfile}" ] && source "${incfile}"
+
+# seems we are not in a dev env
+if [ -z "${BASE_DIR}" ]; then
+	BASE_DIR="$(pwd)"
+	[[ "${BASE_DIR}" == *"/dev" ]] &&  BASE_DIR="${BASE_DIR%/*}"
+	# go to basedir
+	cd "${BASE_DIR}" || exit 1
 fi
 
-#DISTNAME="telegram-bot-bash"
-DISTDIR="./STANDALONE/${DISTNAME}" 
-DISTFILES="bashbot.sh  bashbot.rc commands.sh  mycommands.sh dev/obfuscate.sh modules scripts logs LICENSE README.* doc botacl botconfig.jssh"
+# see if if bashbot is in base dir
+[ ! -f "bashbot.sh" ] && printf "bashbot.sh not found in %s\n" " $(pwd)" && exit 1
 
-# run pre_commit on files
-dev/hooks/pre-commit.sh
+# run pre_commit if exist
+[[ -f "dev/dev.inc.sh"  && "$1" != "--notest" ]] &&  dev/hooks/pre-commit.sh
+
+# files and dirs to copy
+#DISTNAME="telegram-bot-bash"
+DISTDIR="./STANDALONE" 
+DISTMKDIR="data-bot-bash logs bin/logs addons"
+DISTFILES="bashbot.sh commands.sh mycommands.sh modules scripts LICENSE README.* doc addons"
+DISTBINFILES="bin/bashbot_env.inc.sh bin/bashbot_stats.sh bin/process_batch.sh bin/process_update.sh bin/send_broadcast.sh bin/send_message.sh"
+
+# add extra files, minimum mycommands.conf
+extrafile="${BASE_DIR}/dev/${0##*/}.include"
+[ ! -f "${extrafile}" ] && printf "bashbot.rc\nbotacl\nbotconfig.jssh\nmycommands.conf\ndev/obfuscate.sh\n" >"${extrafile}" 
+DISTFILES+=" $(<"${extrafile}")"
 
 # create dir for distribution and copy files
-mkdir -p "${DISTDIR}" 2>/dev/null
+printf "Create directories and copy files\n"
+mkdir -p "${DISTDIR}/bin" 2>/dev/null
 # shellcheck disable=SC2086
-cp -r ${DISTFILES} "${DISTDIR}" 2>/dev/null
+cp -rp ${DISTFILES} "${DISTDIR}" 2>/dev/null
+# shellcheck disable=SC2086
+cp -p ${DISTBINFILES} "${DISTDIR}/bin" 2>/dev/null
+
 cd "${DISTDIR}" || exit 1
 
+# remove log files
+find . -name '*.log' -delete
+
+# shellcheck disable=SC2250
+for dir in $DISTMKDIR
+do
+	[ ! -d "${dir}" ] && mkdir "${dir}"
+done
+
+# inject JSON.sh into distribution
+# shellcheck disable=SC1090
+source "${BASE_DIR}/dev/inject-json.sh"
+
 #######################
 # here the magic starts
 # create all in one bashbot.sh file
 
-echo "OK, now lets do the magic ..."
-echo "    ... create unified commands.sh"
+printf "OK, now lets do the magic ...\n\t... create unified commands.sh\n"
 
 { 
   # first head of commands.sh
-  sed -n '0,/^if / p' commands.sh | head -n -2 
+  sed -n '0,/^if / p' commands.sh | grep -v -F -e "___" -e "*MUST*" -e "mycommands.sh.dist" -e "mycommands.sh.clean"| head -n -2 
 
   # then mycommands from first non comment line on
   printf '\n##############################\n# my commands starts here ...\n'
@@ -53,19 +91,20 @@ echo "    ... create unified commands.sh"
 mv $$commands.sh commands.sh
 rm -f mycommands.sh
 
-echo "    ... create unified bashbot.sh"
+printf "\t... create unified bashbot.sh\n"
 
 { 
   # first head of bashbot.sh
-  sed -n '0,/for modules in/ p' bashbot.sh | head -n -3
+  sed -n '0,/for module in/ p' bashbot.sh | head -n -3
 
-  # then mycommands from first non comment line on
+  # then modules without shebang
   printf '\n##############################\n# bashbot modules starts here ...\n'
-  cat modules/*.sh | sed -e 's/^#\!\/bin\/bash.*//' 
+  # shellcheck disable=SC2016
+  cat modules/*.sh | sed -e 's/^#\!\/bin\/bash.*//' -e '/^#.*\$\$VERSION\$\$/d' 
 
-  # last tail of commands.sh
-  printf '\n##############################\n# bashbot internal functions starts here ...\n\n'
-  sed -n '/BASHBOT INTERNAL functions/,$ p' bashbot.sh
+  # last remaining commands.sh
+  printf '\n##############################\n'
+  sed -n '/^# read commands file/,$ p' bashbot.sh
 
 } >>$$bashbot.sh
 
@@ -74,22 +113,24 @@ chmod +x bashbot.sh
 
 rm -rf modules
 
-echo "Create minimized Version of bashbot.sh and commands.sh"
-sed -E -e '/(shellcheck)|(#!\/bin\/bash)/! s/^[[:space:]]*#.*//' -e 's/^[[:space:]]*//' -e '/^$/d' -e 'N;s/\\\n/ /;P;D' bashbot.sh |\
-	sed 'N;s/\\\n/ /;P;D' > bashbot.sh.min
-sed -E -e '/(shellcheck)|(#!\/bin\/bash)/! s/^[[:space:]]*#.*//' -e 's/^[[:space:]]*//' -e '/^$/d' commands.sh |\
-	sed 'N;s/\\\n/ /;P;D' > commands.sh.min
+printf "Create minimized Version of bashbot.sh and commands.sh\n"
+# shellcheck disable=SC2016
+sed -E -e '/(shellcheck)|(^#!\/)|(\$\$VERSION\$\$)/! s/^[[:space:]]*#.*//' -e '/shellcheck/! s/\t+#.*//' -e 's/^[[:space:]]*//'\
+	 -e '/^$/d' bashbot.sh | sed 'N;s/\\\n/ /;P;D' | sed 'N;s/\\\n/ /;P;D' > bashbot.sh.min
+# shellcheck disable=SC2016
+sed -E -e '/(shellcheck)|(^#!\/)|(\$\$VERSION\$\$)/! s/^[[:space:]]*#.*//' -e '/shellcheck/! s/\t+#.*//' -e 's/^[[:space:]]*//'\
+	  -e '/^$/d' commands.sh | sed 'N;s/\\\n/ /;P;D' > commands.sh.min
 chmod +x bashbot.sh.min
 
 # make html doc
-echo "Create html doc"
+printf "Create html doc\n"
 #shellcheck disable=SC1090
-source "$GIT_DIR/../dev/make-html.sh"
+source "${BASE_DIR}/dev/make-html.sh"
 
-echo "Done!"
+printf "%s Done!\n" "$0"
 
 cd .. || exit 1
 
-echo -e "\\nStandalone bashbot files are now available in \"${DISTDIR}\":\\n"
+printf "\nStandalone bashbot files are now available in %s:\n\n" "${DISTDIR}"
 ls -l "${DISTDIR}"
 

dev/make-standalone.sh.include

@@ -0,0 +1,5 @@
+bashbot.rc
+botacl
+botconfig.jssh
+mycommands.conf
+dev/obfuscate.sh

dev/obfuscate.sh

@@ -1,18 +1,24 @@
 #!/bin/bash
-# shellcheck disable=SC2028,2016
+#
 # joke hack to obfuscate bashbot.min.sh
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+# shellcheck disable=SC2028,SC2016,SC1117
 
 infile="bashbot.sh"
 outfile="./bashbot.obf.sh"
 
-[ ! -f "${infile}" ] && echo "Hey, this is a joke hack to obfuscate ${infile}, copy me to STANDANLONE first" && exit
-
+if [ ! -f "${infile}" ]; then
+	printf "This is a hack to obfuscate %s, run me in STANDALONE after running make-standalone.sh\n" "${infile}"
+	exit 1
+fi
+# create gzipped base64 encoded file plus commands to decode
 {
-echo '#!/bin/bash'
-echo 'a="$PWD";cd "$(mktemp -d)"||exit'
-echo 'printf '"'%s\n'"' '"'$(gzip -9 <bashbot.sh | base64)'"'|base64 -d|gunzip >a;export BASHBOT_HOME="$a";chmod +x a;./a "$@";a="$PWD";cd ..;rm -rf "$a"'
+# shellcheck disable=SC2183
+printf '#!/bin/bash\na="$PWD";cd "$(mktemp -d)"||exit;%s'\
+	'printf '"'$(gzip -9 <"${infile}" | base64)'"'|base64 -d|gunzip >a;export BASHBOT_HOME="$a";chmod +x a;./a "$@";a="$PWD";cd ..;rm -rf "$a"'
 } >"${outfile}"
 
 chmod +x "${outfile}"
 ls -l "${outfile}"
-echo "Try to run ${outfile} init ;-)"
+printf "Try to run %s init ;-)\n" "${outfile}"

dev/shellcheck.files

@@ -1,4 +1,5 @@
 # list of additional files to check from shellcheck
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 bashbot.rc
+mycommands.conf
 mycommands.sh.clean

dev/version.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 # shellcheck disable=SC2016
 #
 # Easy Versioning in git:
@@ -34,34 +34,48 @@
 # run this script to (re)place Version number in files
 #
 
-# magic to ensure that we're always inside the root of our application,
-# no matter from which directory we'll run script
-GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
-if [ "$GIT_DIR" != "" ] ; then
-	cd "$GIT_DIR/.." || exit 1
-else
-	echo "Sorry, no git repository $(pwd)" && exit 1
-fi
+#shellcheck disable=SC1090
+source "${0%/*}/dev.inc.sh"
 
 unset IFS
 # set -f # if you are paranoid use set -f to disable globbing
 
 VERSION="$(git describe --tags --long)"
-echo "Update to version $VERSION ..."
+printf "Update to version %s ...\n" "${VERSION}"
 
-FILES="$(find ./*)"
-[ "$1" != "" ] && FILES="$*"
+# only regular files, ignore .dot files/dirs, e.g. .git .gitinore in BASEDIR
+if [ -n "$1" ]; then
+	FILES="$*"
+else
+	printf "Update version string in all files? (y/N)\b\b"
+	read -r answer
+	[[ "${answer}" != "y" && "${answer}" != "Y" ]] && exit
+	FILES="$(find ./* -type f ! -path "./DIST/*" ! -path "./STANDALONE/*")"
+fi
 
-for file in $FILES
+# autogenerate REMADME.html REMADE.txt
+if [[ "${FILES}" == *"README.md"* ]]; then
+	FILES+=" README.html README.txt"
+	type -f pandoc >/dev/null && pandoc -s -f commonmark -M "title=Bashbot README" README.md >README.html
+	cat "doc/bashbot.ascii" >"README.txt"
+	if [ -r "README.html" ] && type -f html2text >/dev/null; then
+		# convert html links to text [link]
+		sed -E 's/<a href="([^>]+)">([^<#]+)<\/a>/\2 [\1]/g' <README.html |\
+		html2text -style pretty -width 90 - >>README.txt
+	else
+		type -f fold >/dev/null && fold -s -w 90 README.md >>README.txt
+	fi
+fi
+
+# change version string in given files
+for file in ${FILES}
 do
-	[ ! -f "$file" ] && continue
-	#[ "$file" == "version" ] && continue
-	echo -n " $file" >&2
-	sed -i 's/^#### $$VERSION$$.*/#### \$\$VERSION\$\$ '"$VERSION"'/' "$file"
+	# symlink is a file :-(
+	[[  -L "${file}" || ! -f "${file}" ]] && continue
+	#[ "${file}" == "version" ] && continue
+	printf "%s" " ${file}" >&2
+	sed -i 's/^#### $$VERSION$$.*/#### \$\$VERSION\$\$ '"${VERSION}"'/' "${file}"
 done
-# try to compile README.txt
-echo -n " README.txt" >&2
-type -f pandoc >/dev/null && pandoc -s -f commonmark -M "title=Bashbot README" README.md >README.html
-fold -s README.md >README.txt
-echo " done."
+
+printf " done.\n"
 

doc/0_install.md

@@ -2,7 +2,9 @@
 
 ## Check bash installation
 
-There may systems where bash seems to be installed but it is not, e.g. embedded systems, or where bash is to old.
+There may systems where bash seems to be installed but it is not (_e.g. embedded systems_) or where bash is to old.
+Bashbot has some builtin checks but it may better to check before installing bashbot.
+
 Run the following commands to see if your bash looks ok ...
 
 ```bash
@@ -13,72 +15,90 @@ if which bash; then echo "bash seems available..."; else echo "NO bash"; fi
 bash -c 'if eval "a[1]=1"; then echo "Shell support arrays..."; else echo "Shell has NO arrays"; fi'
 
 # check for bash version by feature
-bash -c 'if [ "$(echo -e "\u1111")" != "\u1111" ]; then echo "Bash version ok ..."; else echo "LBash Version may be to old ..."; fi'
+bash -c 'if [ "$(LANG=C.UTF-8 echo -e "\u1111")" != "\u1111" ]; then echo "Bash version ok ..."; else echo "Bash version may to old ..."; fi'
+
+# display bash version, must be greater than 4.3
+bash --version | grep "bash"
 ```
 
 ## Install bashbot
 
-1. Go to the directory you want to install bashbot, e.g.
-    * your $HOME directory (install and run with your user-ID)
-    * /usr/local if you want to run as service
-2. [Download latest release zip / tar archive from github](https://github.com/topkecleon/telegram-bot-bash/releases) and extract all files. 
-3. Change into the directory ```telegram-bot-bash```
-4. Activate the bot example commands  ```cp mycommands.sh.dist mycommands.sh```
-5. Run ```./bashbot.sh init``` to setup the environment and enter your Bots token given by botfather.
+Installing bashbot is very simple: Download and extract the installation archive.
 
-Edit ```mycommands.sh``` to your needs.
+1. Choose a directory to install bashbot (_e.g.your HOME or /usr/local_)
+2. Download [latest release zip / tar archive](https://github.com/topkecleon/telegram-bot-bash/releases/latest) and extract all files. 
+3. Change into the directory `telegram-bot-bash`
+4. Copy `mycommands.conf.dist` `mycommands.conf`
+4. Copy `mycommands.sh.dist` or `mycommands.sh.clean` to `mycommands.sh`
+5. Run `./bashbot.sh init`\* to setup the environment and enter your Bots token given by botfather.
+
+Edit config in `mycommands.conf` and commands in `mycommands.sh` to fit your need.
 Now your Bot is ready to start ...
 
-**If you are new to Bot development read [Bots: An introduction for developers](https://core.telegram.org/bots)**
+*If you are new to Bot development read [Bots: An introduction for developers](https://core.telegram.org/bots)*
 
+\* _Run with sudo if you want to run bashbot from different user, e.g. from `bashbot.rc`._
 
 ### Update bashbot
 
-**Note: all files including ```mycommands.sh``` may overwritten, make a backup!**
+Update bashbot is almost identical to installing bashbot: Download and extract the installation archive.
 
-1. Go to the directory where you had installed bashbot, e.g.
-    * your $HOME directory
-    * /usr/local
-2. [Download latest release zip / tar archive from github](https://github.com/topkecleon/telegram-bot-bash/releases)
-3. Stop all running instances of bashbot
-4. Extract all files to your existing bashbot dir 
-5. Run ```sudo ./bashbot.sh init``` to setup your environment after the update
+**Important: All files may overwritten, make a backup!**
 
-If you modified ```commands.sh``` move your changes to ```mycommands.sh```, this avoids overwriting your commands on update.
+1. Go to the directory where bashbot is installed (_e.g.$HOME/telegram-bot-bash or /usr/local/telegram-bot-bash_)
+2. Download [latest release zip / tar archive](https://github.com/topkecleon/telegram-bot-bash/releases/latest)
+3. Stop all running instances of bashbot `./bashbot.sh stop`
+4. Change to parent directory of bashbot installation and extract all files from archive.
+5. Run `./bashbot.sh init`\* to setup your environment after the update
+6. Restart your bot `./bashbot.sh start`
 
-Now you can restart your bashbot instances.
+`mycommands.conf` and `mycommands.sh` will not overwritten, this avoids losing your bot config and commands on updates.
 
-### Get zip / tar archive on command line
+*Note*: If you are updating from a pre-1.0 version, update to [Version 1.20](https://github.com/topkecleon/telegram-bot-bash/releases/tags/v1.20) first!
 
-```bash
-wget -q https://github.com/$(wget -q https://github.com/topkecleon/telegram-bot-bash/releases/latest -O - | egrep '/.*/.*/.*tar.gz' -o)
-```
+### Use JSON.awk
+
+[JSON.awk](https://github.com/step-/JSON.awk) is an awk port of `JSON.sh`, it provides the same functionality but is 5 times faster.
+On most systems you can use `JSON.awk` with system default awk installation.
+( [gnu awk, posix awk, mawk, busybox akw](https://github.com/step-/JSON.awk#compatibility-with-awk-implementations) ).
+
+After you have checked that `JSON.awk.dist` is working on your system copy it to `JSON.awk` and (re)start bashbot.
+
+BSD and MacOS users must install `gnu awk` and adjust the shebang, see below
+
+*Note*: To install or update `JSON.awk` manually execute the following commands in the directory `JSON.sh/`:
+
+	wget https://cdn.jsdelivr.net/gh/step-/JSON.awk/JSON.awk 
+	wget https://cdn.jsdelivr.net/gh/step-/JSON.awk/tool/patch-for-busybox-awk.sh
+	bash patch-for-busybox-awk.sh
+	chmod +x JSON.awk
 
 
-### Install from github
+### Install bashbot from git repo
 
-The first install can also be done from github, updates should always done from the zip or tar archive to avoid
-overwriting your config and ```mycommands.sh``` files.
+Installation and updates should be done using the zip / tar archives provided on github to avoid
+problems and not overwriting your bot config and `mycommands.sh`.
 
-1. Go to the directory you want to install bashbot, e.g.
-    * your $HOME directory (install and run with your user-ID)
-    * /usr/local if you want to run as service
-2. Run ```git clone https://github.com/topkecleon/telegram-bot-bash.git```
-3. Change into the directory ```telegram-bot-bash```
-4. Run ``` dev/all-tests.sh``` and if everything finish OK ...
-5. Run ``` rm addons/*``` to deactivate the example add on's
-5. Run ```sudo ./bashbot.sh init``` to setup the environment and enter your Bots token given by botfather.
+Nevertheless you can install or update bashbot from a git repo, see next chapter ...
 
 
-### Get the latest development updates from github
+### Create Installation / Update archives
 
-To update an existing installation to the latest development version from github you must create the update archives yourself.
+To install or update bashbot from git repo execute `dev/make-distribution.sh`.
+This creates the installation archives in `DIST/` and a ready to run test installation in `DIST/telegram.bot-bash`.
 
-1. Run ```git clone https://github.com/topkecleon/telegram-bot-bash.git```
-2. Change into the directory ```telegram-bot-bash```
-3. Run ``` git checkout develop```
-4. Run ``` dev/make-distribution.sh```
-5. If no errors occur you will find archive files in DISTRIBUTION
+*Note:* You should be familiar with `git`.
+
+1. Run `git clone https://github.com/topkecleon/telegram-bot-bash.git`
+2. Change into the directory `telegram-bot-bash`
+3. Optional: Run ` git checkout develop` for latest develop version
+4. Run ` dev/make-distribution.sh` (_add --notest to skip tests_)
+5. Change to dir `DIST/`
+
+Use the installation archives to install or update bashbot as described above.
+
+To run a test bot, e.g. while development or testing latest changes, you can use the bashbot installation provided in `DIST/telegram-bot-bash`.
+To update the test installation (_after git pull, local changes or switch master/develop_) run `dev/make-distribution.sh` again.
 
 
 ### Note for BSD and MacOS
@@ -87,87 +107,30 @@ To update an existing installation to the latest development version from github
 see e.g. [Install Bash on Mac](http://macappstore.org/bash/)
 
 **On BSD and MacOS** I recommend to install gnu coreutils and include them in your PATH
-environment variable before running bashbot, e.g. the gnu versions of sed, grep, find ...
+environment variable before running bashbot, e.g. the gnu versions of sed, grep, find, awk ...
 
-On BSD and MacOS you must adjust the shebang line of the scripts ```bashbot.sh``` and ```json.sh``` to point to to the correct bash
-or use the script: ```examples/bash2env *.sh */*.sh``` to convert them for you.
+On BSD and MacOS you must adjust the shebang line of the scripts `bashbot.sh` and `json.sh` to point to to the correct bash
+or use the script: `examples/bash2env *.sh */*.sh` to convert them for you.
 
-Bashbot will stay with /bin/bash shebang, as using a fixed path is more secure than the portable /usr/bin/env variant, see
-[Security Considerations](../README.md#Security-Considerations)
+Bashbot will stay with `#!/bin/bash` shebang, as using a fixed path is IMHO more secure than the portable '!/usr/bin/env bash` variant.
 
-I considered to make bashbot BSD sed compatible, but much of the bashbot "magic" relies on
-(gnu) sed features, e.g. alternation ```|```, non printables ```\n\t\<``` or repeat ```?+``` pattern, not supported by BSD sed.
-BSD/MacOS sed compatibility will result in a rewrite of all grep/sed commands with an uncertain outcome,
+Compatibility with BSD/MacOS will result in a rewrite of all grep/sed commands with an uncertain outcome,
 see [BSD/MacOS vs. GNU sed](https://riptutorial.com/sed/topic/9436/bsd-macos-sed-vs--gnu-sed-vs--the-posix-sed-specification)
 to get an impression how different they are.
 
-If you can convert the following examples to work correct with gnu and BSD sed, contact me.
 
-```bash
-  # easy start
-  sed -n -e '0,/\['"$1"'\]/ s/\['"$1"'\][ \t]\([0-9.,]*\).*/\1/p'
-  OUT="$(sed -e ':a;N;$!ba;s/\r\n/ mynewlinestartshere /g' <<<"$1"| iconv -f utf-8 -t utf-8 -c)"
+### Notes on Changes
 
-  # more complex
-  address="$(sed <<< "${2}" '/myaddressstartshere /!d;s/.*myaddressstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+#### Config moved to mycommands.conf
 
-  # for experts?
-  source <( printf "$1"'=( %s )' "$(sed -E -n -e ':x /"text"\]/ { N; s/([^"])\n/\1\\n/g ; tx }' -e '/\["[-0-9a-zA-Z_,."]+"\]\+*\t/ s/\t/=/gp' -e 's/=(true|false)/="\1"/')" )
-```
-
-### Notes per Version
-
-#### Change in storing config values
-
-Up to version 0.94 bashbot stores config values as values in ```token```, ```botadmin``` and ```count```. Since version 0.96 bashbot
-uses jsonDB key/value store. Config is stored in ```botconfig.jssh```, counting of users is done in ```count.jssh```.
-The acl file ```botacl``` stay as is. On first run of bashbot.sh after an update bashbot converts
-the files to the new config format. Afterwards the files ```token```, ```botadmin``` and ```count``` can be deleted.
-
-You may notice the new file ```blocked.jssh```, every telegram user or chat id stored here will be blocked from 
-using your bot.
-
-#### removal of TMUX
-From version 0.80 on TMUX is no longer needed and the bashbot command 'attach' is deleted. Old function 'inproc'
-is replaced by 'send_interactive'. send_interactive does checks if an interactive job is running internally.
-Pls check if you make use of inproc and remove it including the old checks, e.g.
-```bash
-if tmux ls | grep -v send | grep -q "$copname"; then inproc; fi
-# or
-[ checkprog ] && inproc
-```
-must be replaced by ```send_interactive "${CHATD[ID]}" "${MESSAGE}"```
-
-### Do not edit commands.sh
-From version 0.60 on your commands must be placed in ```mycommands.sh```. If you update from a version with your commands
-in 'commands.sh' move all your commands and functions to ```mycommands.sh```.
-
-From version 0.80 on 'commands.sh' will be overwritten on update!
-
-#### Location of var / tmp / data dirs
-From version 0.70 on the tmp dir is renamed to 'data-bot-bash' to reflect the fact that not only temporary files are stored. an existing 'tmp-bot-bash' will be automatically renamed after update.
-
-From version 0.50 on the temporary files are no more placed in '/tmp'. instead a dedicated tmp dir is used.
-
-#### Changes to send_keyboard in v0.6
-From Version 0.60 on keyboard format for ```send_keyboard``` and ```send_message "mykeyboardstartshere ..."``` was changed.
-Keyboards are now defined in JSON Array notation e.g. "[ \\"yes\\" , \\"no\\" ]".
-This has the advantage that you can create any type of keyboard supported by Telegram.
-The old format is supported for backward compatibility, but may fail for corner cases.
-
-*Example Keyboards*:
-
-- yes no in two rows:
-    - OLD format: 'yes' 'no' *(two strings)*
-    - NEW format: '[ "yes" ] , [ "no" ]' *(two arrays with a string)*
-- new layouts made easy with NEW format:
-    - Yes No in one row: '[ "yes" , "no" ]'
-    - Yes No plus Maybe in 2.row: '[ "yes" , "no" ] , [ "maybe" ]' 
-    - numpad style keyboard: '[ "1" , "2" , "3" ] , [ "4" , "5" , "6" ] , [ "7" , "8" , "9" ] , [ "0" ]'
+From Version 1.30 on config for new bots is moved to `mycommands.conf`.
 
+#### Support for update from pre-1.0 removed
 
+From Version 1.21 on updating from a pre-1.0 version (_no \*.jssh config_) is no more supported!
+You must update to [Version 1.20](https://github.com/topkecleon/telegram-bot-bash/releases/tags/v1.20) first!
 
 #### [Next Create Bot](1_firstbot.md)
 
-#### $$VERSION$$ v1.1-0-ge835bbc
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/1_firstbot.md

@@ -65,5 +65,5 @@ group. This step is up to you actually.
 #### [Prev Installation](0_install.md)
 #### [Next Getting started](2_usage.md)
 
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/2_usage.md

@@ -1,14 +1,15 @@
 #### [Home](../README.md)
 ## Getting Started
 
-The Bots standard commands are in the commands dispatcher ```commands.sh```, Do not edit this file! Add your commands and functions to ```mycommands.sh```. In 'mycommands.sh.dist' you find examples how to add own commands and overwrite existing ones. See [Best practices](5_practice.md) for more information.
+The Bots default commands are in `commands.sh`. Do not edit this file! Instead copy `mycommands.sh.clean` to `mycommands.sh` and place you commands there.
+Have a look at `mycommands.sh.dist` for examples on how to write commands or overwrite existing ones. See [Best practices](5_practice.md) for more information.
 
-Once you're done with editing start the Bot with ```./bashbot.sh start```.  To stop the Bot run ```./bashbot.sh stop```
+Once you're done with editing run the Bot with `./bashbot.sh start`.  To stop running the Bot use `./bashbot.sh stop`
 
-If something doesn't work as expected, debug with ```./bashbot.sh startbot DEBUG &```, where DEBUG can be 'debug', 'xdebug' or 'xdebugx'.
+If something doesn't work as expected, debug with `./bashbot.sh startbot DEBUG &`, where DEBUG can be 'debug', 'xdebug' or 'xdebugx'.
 See [Bashbot Development](7_develop.md) for more information.
 
-To use the functions provided in this script in other scripts simply source bashbot: ```source bashbot.sh source```. see [Expert Use](8_expert.md#Expert-use)
+To use the functions provided in this script in other scripts simply source bashbot: `source bashbot.sh source`. see [Expert Use](8_expert.md#Expert-use)
 
 Have FUN!
 
@@ -18,8 +19,10 @@ Have FUN!
 ```
 .
 ├── mycommands.sh        # THIS is your bot, place logic and commands here!
+├── mycommands.conf      # place your bot config and bot messages here!
 │
-├── mycommands.sh.clean      # copy to "mycommands.sh" if you start developing your bot
+├── mycommands.conf.dist     # copy to "mycommands.conf" if not exist
+├── mycommands.sh.clean      # copy to "mycommands.sh" to start developing a new bot
 ├── mycommands.sh.dist       # example bot, also used for testing bashbot internally 
 │
 ├── count.jssh           # count bashbot usage in jssh key-value store
@@ -27,7 +30,24 @@ Have FUN!
 │
 ├── bashbot.sh           # main bashbot script - DO NOT EDIT!
 ├── commands.sh          # command dispatcher - DO NOT EDIT!
-├── JSON.sh              # bashbots JSON parser, see https://github.com/dominictarr/JSON.sh
+├── JSON.sh              # bashbot JSON parsers
+│   ├── JSON.sh              # sh implementation, https://github.com/dominictarr/JSON.sh
+│   └── JSON.awk.dist        # faster awk version, https://github.com/step-/JSON.awk
+│
+├── bin                  # ready to use scripts, use `scriptname --help` for help
+│   ├── bashbot_stats.sh         # does what it says ...
+│   ├── send_broadcast.sh        # send message to all known chats
+│   ├── send_message.sh          # send message to given chat
+│   ├── edit_message.sh          # replace given message id in given chat
+│   ├── send_file.sh             # send file to given chat
+│   ├── delete_message.sh        # delete given message id in given chat
+│   ├── send_buttons.sh          # send message with attached button
+│   ├── edit_buttons.sh          # attach/edit message buttons
+│   ├── kickban_user.sh          # kick/unban user from given chat
+│   ├── promote_user.sh          # promote/dente user rights in given chat
+│   │
+│   ├── bashbot_env.inc.sh       # sourced from scripts, adapt locations if needed
+│   └── bashbot_init.inc.sh      # sourced from bashbot.sh init
 │
 ├── scripts              # place your bashbot interactive and background scripts here
 │   └── interactive.sh.clean     # interactive script template for new scripts
@@ -37,15 +57,15 @@ Have FUN!
 ├── modules              # optional functions, sourced by commands.sh
 │   ├── aliases.sh           # to disable modules rename them xxx.sh.off
 │   ├── answerInline.sh
-│   ├── jsshDB.sh            # read and store JSON.sh style JSON, mandatory
 │   ├── background.sh        # interactive and background functions
-│   ├── chatMember.sh
-│   └── sendMessage.sh       # main send message functions, mandatory
+│   ├── chatMember.sh        # manage chat mambers
+│   ├── jsshDB.sh            # read and store JSON.sh style JSON, mandatory
+│   ├── processUpdates.sh    # process updates from telegram, mandatory (run bot)
+│   └── sendMessage.sh       # send message functions, mandatory
 │
 ├── addons               # optional addons, disabled by default
 │   ├── example.sh           # to enable addons change their XXX_ENABLE to true
-│   ├── antiFlood.sh         # simple addon taking actions based on # files and text sent to chat
-│   └── xxxxxage.sh
+│   └── antiFlood.sh         # simple addon taking actions based on # files and text sent to chat
 │
 ├── bashbot.rc           # start/stop script if you run bashbot as service
 │
@@ -77,142 +97,230 @@ Start or Stop your Bot use the following commands:
 ./bashbot.sh stop
 ```
 
-### User stats
+### Scripts in bin/
+Use `script.sh -h` or `script --help` to get short/long help for script.
 
 To count the total number of users and messages run the following command:
 
+```bash
+bin/bashbot_stats.sh
 ```
-./bashbot.sh stats
-```
-
-### Sending broadcasts to all users
 
 To send a broadcast to all of users that ever used the bot run the following command:
 
+```bash
+bin/send_broadcast.sh "Hey, I just wanted to let you know that the bot's been updated!"
+
+Sending broadcast message to all users of Deal_O_Mat_bot
+DRY RUN! use --doit as first argument to execute broadcast...
+...
+Message "Hey, ..." sent to xxx users.
 ```
-./bashbot.sh broadcast "Hey! I just wanted to let you know that the bot's been updated!"
+
+To send a message to one user or chat run the following command:
+
+```bash
+bin/send_message.sh "CHAT[ID]" "Hey, I just wanted to let you know that the bot's been updated!"
+
+["OK"]  "true"
+["ID"]  "12345"
 ```
 
+To replace a message already sent to one user or chat run the following command:
+
+```bash
+bin/edit_message.sh "CHAT[ID]" "12345" "Done!"
+
+["OK"]  "true"
+["ID"]  "12345"
+```
+
+To send a file to one user or chat run the following command:
+
+```bash
+bin/send_file.sh "CHAT[ID]" "funny-pic.jpg" "enjoy this picture"
+
+["OK"]  "true"
+["ID"]  "12346"
+```
+
+Note: to get help about a script in bin/ run `scriptname.sh --help`
+
 ----
 
 ## Receive data
-Evertime a Message is received, you can read incoming data using the following variables:
+Evertime a Telegram update is received, you can read incoming data using the following variables:
+In case you need other update values, the array `UPD` contains complete Telegram response.
 
-### Regular Messages
+### Processing Messages
 
-These Variables are always present in regular messages:
+If an update is received from Telegram, the message is pre processed by Bashbot and the following bash variables are set for use in `mycommands.sh`.
 
-* ```${MESSAGE}```: Current message text
-* ```${MESSAGE[ID]}```: ID of current message
-* ```$USER```: This array contains the First name, last name, username and user id of the sender of the current message.
-    * ```${USER[ID]}```: User id
-    * ```${USER[FIRST_NAME]}```: User's first name
-    * ```${USER[LAST_NAME]}```: User's last name
-    * ```${USER[USERNAME]}```: Username
-* ```$CHAT```: This array contains the First name, last name, username, title and user id of the chat of the current message.
-    * ```${CHAT[ID]}```: Chat id
-    * ```${CHAT[FIRST_NAME]}```: Chat's first name
-    * ```${CHAT[LAST_NAME]}```: Chat's last name
-    * ```${CHAT[USERNAME]}```: Username
-    * ```${CHAT[TITLE]}```: Title
-    * ```${CHAT[TYPE]}```: Type
-    * ```${CHAT[ALL_MEMBERS_ARE_ADMINISTRATORS]}```: All members are administrators (true if true)
+These variables are always present if a message is pre processed:
+
+* `${ME}`: Name of your bot
+* `${BOTADMIN}`: User id of bot administrator
+
+* `${MESSAGE}`: Current message text
+* `${MESSAGE[ID]}`: ID of current message
+* `$USER`: This array contains the First name, last name, username and user id of the sender of the current message.
+    * `${USER[ID]}`: User id
+    * `${USER[FIRST_NAME]}`: User's first name
+    * `${USER[LAST_NAME]}`: User's last name
+    * `${USER[USERNAME]}`: Username
+* `$CHAT`: This array contains the First name, last name, username, title and user id of the chat of the current message.
+    * `${CHAT[ID]}`: Chat id
+    * `${CHAT[FIRST_NAME]}`: Chat's first name
+    * `${CHAT[LAST_NAME]}`: Chat's last name
+    * `${CHAT[USERNAME]}`: Username
+    * `${CHAT[TITLE]}`: Title
+    * `${CHAT[TYPE]}`: Type
+    * `${CHAT[ALL_MEMBERS_ARE_ADMINISTRATORS]}`: All members are administrators (true if true)
 
 The following variables are set if the message contains optional parts:
 
-* ```$REPLYTO```: Original message which was replied to
-* ```$REPLYTO```: This array contains the First name, last name, username and user id of the ORIGINAL sender of the message REPLIED to.
-    * ```${REPLYTO[ID]}```: ID of message which was replied to
-    * ```${REPLYTO[UID]}```: Original user's id
-    * ```${REPLYTO[FIRST_NAME]}```: Original user's first name
-    * ```${REPLYTO[LAST_NAME]}```: Original user's' last name
-    * ```${REPLYTO[USERNAME]}```: Original user's username
-* ```$FORWARD```: This array contains the First name, last name, username and user id of the ORIGINAL sender of the FORWARDED message.
-    * ```${FORWARD[ID]}```: Same as MESSAGE[ID] if message is forwarded
-    * ```${FORWARD[UID]}```: Original user's id
-    * ```${FORWARD[FIRST_NAME]}```: Original user's first name
-    * ```${FORWARD[LAST_NAME]}```: Original user's' last name
-    * ```${FORWARD[USERNAME]}```: Original user's username
-* ```$CAPTION```: Picture, Audio, Video, File Captions
-* ```$URLS```: This array contains documents, audio files, voice recordings and stickers as URL.
-    * ```${URLS[AUDIO]}```: Audio files
-    * ```${URLS[VIDEO]}```: Videos
-    * ```${URLS[PHOTO]}```: Photos (maximum quality)
-    * ```${URLS[VOICE]}```: Voice recordings
-    * ```${URLS[STICKER]}```: Stickers
-    * ```${URLS[DOCUMENT]}```: Any other file
-* ```$CONTACT```: This array contains info about contacts sent in a chat.
-    * ```${CONTACT[ID]}```: User id
-    * ```${CONTACT[NUMBER]}```: Phone number
-    * ```${CONTACT[FIRST_NAME]}```: First name
-    * ```${CONTACT[LAST_NAME]}```: Last name
-    * ```${CONTACT[VCARD]}```: User's complete Vcard
-* ```$LOCATION```: This array contains info about locations sent in a chat.
-    * ```${LOCATION[LONGITUDE]}```: Longitude
-    * ```${LOCATION[LATITUDE]}```: Latitude
-* ```$VENUE```: This array contains info about venue (a place) sent in a chat.
-    * ```${VENUE[TITLE]}```: Name of the place
-    * ```${VENUE[ADDRESS]}```: Address of the place
-    * ```${VENUE[LONGITUDE]}```: Longitude
-    * ```${VENUE[LATITUDE]}```: Latitude
-    * ```${VENUE[FOURSQUARE]}```: Fouresquare ID
+* `MESSAGE[CAPTION]`: Picture, Audio, Video, File Captions
+* `MESSAGE[DICE]`: Animated DICE Emoji DICE values is contained in `MESSAGE[RESULT]`
+* `$REPLYTO`: Original message which was replied to
+* `$REPLYTO`: This array contains the First name, last name, username and user id of the ORIGINAL sender of the message REPLIED to.
+    * `${REPLYTO[ID]}`: ID of message which was replied to
+    * `${REPLYTO[UID]}`: Original user's id
+    * `${REPLYTO[FIRST_NAME]}`: Original user's first name
+    * `${REPLYTO[LAST_NAME]}`: Original user's' last name
+    * `${REPLYTO[USERNAME]}`: Original user's username
+* `$FORWARD`: This array contains the First name, last name, username and user id of the ORIGINAL sender of the FORWARDED message.
+    * `${FORWARD[ID]}`: Same as MESSAGE[ID] if message is forwarded
+    * `${FORWARD[UID]}`: Original user's id
+    * `${FORWARD[FIRST_NAME]}`: Original user's first name
+    * `${FORWARD[LAST_NAME]}`: Original user's' last name
+    * `${FORWARD[USERNAME]}`: Original user's username
+* `$URLS`: This array contains the `path` on Telegram server for files send to chat, e.g. photo, video, audio file.
+    * `${URLS[AUDIO]}`: Path to audio file
+    * `${URLS[VIDEO]}`: Path to video
+    * `${URLS[PHOTO]}`: Path to photo (maximum quality)
+    * `${URLS[VOICE]}`: Path to voice recording
+    * `${URLS[STICKER]}`: Path to sticker
+    * `${URLS[DOCUMENT]}`: Path to any other file
+**Important:** This is NOT a full URL, you must use `download_file "${URLS[xxx]}"` or prefix path with telegram api url for manual download
+(_e.g. `getJson "${URL}/${URLS[xxx]}" >file`_).
+* `$CONTACT`: This array contains info about contacts sent in a chat.
+    * `${CONTACT[ID]}`: User id
+    * `${CONTACT[NUMBER]}`: Phone number
+    * `${CONTACT[FIRST_NAME]}`: First name
+    * `${CONTACT[LAST_NAME]}`: Last name
+    * `${CONTACT[VCARD]}`: User's complete Vcard
+* `$LOCATION`: This array contains info about locations sent in a chat.
+    * `${LOCATION[LONGITUDE]}`: Longitude
+    * `${LOCATION[LATITUDE]}`: Latitude
+* `$VENUE`: This array contains info about venue (a place) sent in a chat.
+    * `${VENUE[TITLE]}`: Name of the place
+    * `${VENUE[ADDRESS]}`: Address of the place
+    * `${VENUE[LONGITUDE]}`: Longitude
+    * `${VENUE[LATITUDE]}`: Latitude
+    * `${VENUE[FOURSQUARE]}`: Fouresquare ID
 
 
 ### Service Messages
 
-Service Messages are regular messages not itended for end users, instead they signal special events to the
-client, e.g. new users.
+Service Messages are updates not itended for end users, instead they signal special events in a chat, e.g. new users.
 
-If a service message is received bashbot sets MESSAGE to the service message type as a command,
-e.g. if a new user joins a chat MESSAGE is set to "/_new_chat_user". 
+If a service message is received bashbot pre processing sets `${MESSAGE}` according to the service message type,
+e.g. if a new user joins a chat MESSAGE is set to `/_new_chat_user ...`. 
 
-* ```$SERVICE```: This array contains info about received service messages.
-    * ```${SERVICE}```: "yes" if service message is received
-    * ```${SERVICE[NEWMEMBER]}}```: New user's id
-        * ```${MESSAGE}```: /_new_chat_member ID NAME
-        * ```${NEWMEMBER[ID]}```: New user's id
-        * ```${NEWMEMBER[FIRST_NAME]}```: New user's first name
-        * ```${NEWMEMBER[LAST_NAME]}```: New user's last name
-        * ```${NEWMEMBER[USERNAME]}```: New user's username
-        * ```${NEWMEMBER[ISBOT]}```: New user is a bot
-    * ```${SERVICE[LEFTMEMBER]}```: Id of user left 
-        * ```${MESSAGE}```: /_left_chat_member ID NAME
-        * ```${LEFTMEMBER[ID]}```: Left user's id
-        * ```${LEFTMEMBER[FIRST_NAME]}```: Left user's first name
-        * ```${LEFTMEMBER[LAST_NAME]}```: Left user's last name
-        * ```${LEFTMEMBER[USERNAME]}```: Left user's username
-        * ```${LEFTMEMBER[ISBOT]}```: Left user is a bot
-    * ```${SERVICE[NEWTITLE]}```: Text of new title 
-        * ```${MESSAGE}```: /_new_chat_title SENDER TEXT
-    * ```${SERVICE[NEWPHOTO]}```: New Chat Picture 
-        * ```${MESSAGE}```: /_new_chat_picture SENDER URL
-    * ```${SERVICE[PINNED]}```: Pinned Message structure
-        * ```${MESSAGE}```: /_new_pinned_message SENDER ID
-        * ```${PINNED[ID]}```: Id of pinned message
-        * ```${PINNED[MESSAGE]}```: Message text of pinned message
+* `$SERVICE`: This array contains info about received service messages.
+    * `${SERVICE}`: "yes" if service message is received
+    * `${SERVICE[NEWMEMBER]}}`: New user's id
+        * `${MESSAGE}`: /_new_chat_member ID NAME
+        * `${NEWMEMBER[ID]}`: New user's id
+        * `${NEWMEMBER[FIRST_NAME]}`: New user's first name
+        * `${NEWMEMBER[LAST_NAME]}`: New user's last name
+        * `${NEWMEMBER[USERNAME]}`: New user's username
+        * `${NEWMEMBER[ISBOT]}`: New user is a bot
+    * `${SERVICE[LEFTMEMBER]}`: Id of user left 
+        * `${MESSAGE}`: /_left_chat_member ID NAME
+        * `${LEFTMEMBER[ID]}`: Left user's id
+        * `${LEFTMEMBER[FIRST_NAME]}`: Left user's first name
+        * `${LEFTMEMBER[LAST_NAME]}`: Left user's last name
+        * `${LEFTMEMBER[USERNAME]}`: Left user's username
+        * `${LEFTMEMBER[ISBOT]}`: Left user is a bot
+    * `${SERVICE[NEWTITLE]}`: Text of new title 
+        * `${MESSAGE}`: /_new_chat_title SENDER TEXT
+    * `${SERVICE[NEWPHOTO]}`: New Chat Picture 
+        * `${MESSAGE}`: /_new_chat_picture SENDER URL
+**Important:** SERVICE[NEWPHOTO] is NOT a full URL, you must use `download_file "${SERVICE[NEWPHOTO]}"` or prefix path with telegram api url for manual download
+(_e.g. `getJson "${FILEURL}/${SERVICE[NEWPHOTO]}" >file`_).
+    * `${SERVICE[PINNED]}`: Pinned MESSAGE ID
+        * `${MESSAGE}`: /_new_pinned_message SENDER ID
+        * `${PINNED[ID]}`: Id of pinned message
+        * `${PINNED[MESSAGE]}`: Message text of pinned message
+    * `${SERVICE[MIGRATE]}`: Old and new group id
+        * `${MESSAGE}`: /_migrate_group MIGRATE_FROM MIGRATE_TO
+        * `${MIGRATE[FROM]}`: Old group id
+        * `${MIGRATE[TO]}`: New group id
 
 
 
 ### Inline query messages
+Inline query messages are special messages for direct interaction with your bot.
+If an user starts an inline conversation an inline query is sent after each user keystroke.
+
+To receive inline messages you must set `inline=1` in `mycommands.conf` and in botfather.
+THe message contatains all characters so far typed from the user.
+
+An received inline query must be anserwered with `answer_inline_query`, see also (Inline Query)[6_reference.md#inline-query]
+
+If an inline query is received only the following variables are available:
+
+* `${iQUERY}`: Inline message typed so far by user
+* `$iQUERY`: This array contains the ID, First name, last name, username and user id of the sender of the current inline query.
+    * `${iQUERY[ID]}`: Inline query ID
+    * `${iQUERY[USER_ID]}`: User's id
+    * `${iQUERY[FIRST_NAME]}`: User's first name
+    * `${iQUERY[LAST_NAME]}`: User's last name
+
+
+
+### Callback button messages
+Callback button messages special messages swend from callback buttons, they contain the following variables only:
+
+* `$iBUTTON`: This array contains the ID, First name, last name, username and user id of the user clicked on the button
+    * `${iBUTTON[ID]}`: Callback query ID
+    * `${iBUTTON[DATA]`: Data attached to button, hopefully unique
+    * `${iBUTTON[CHAT_ID]`: Chat where button was pressed
+    * `${iBUTTON[MESSAGE_ID]`: Message to which button is attached
+    * `${iBUTTON[MESSAGE]`: Text of message
+    * `${iBUTTON[USER_ID]}`: User's id
+    * `${iBUTTON[FIRST_NAME]}`: User's first name
+    * `${iBUTTON[LAST_NAME]}`: User's last name
+    * `${iBUTTON[USERNAME]}`: User's @username
+
+
+## Send data / get response
+
+After every `send_xxx` `get_xxx` call the array BOTSENT contains the most important values from Telegram response.
+In case you need other response values , the array `UPD` contains complete Telegram response.
+
+You can use the array values to check if a commands was successful and get returned values from Telegram.
+
+### BOTSENT array
+
+* `$BOTSENT`: This array contains the parsed results from the last transmission to telegram.
+    * `${BOTSENT[OK]}`: contains the string `true`: after a successful transmission
+    * `${BOTSENT[ID]}`: Message ID of sent message, image, file etc., if OK is true
+    * `${BOTSENT[FILE_ID]}`: unique identifier returned for an uploaded file or URL
+    * `${BOTSENT[FILE_TYPE]}`: file type: photo, audio, video, sticker, voice, document
 
-Inline query messages are small, non regular messages used for interaction with the user,
-they contain the following variables only:
 
-* ```${iQUERY}```: Current inline query
-* ```$iQUERY```: This array contains the ID, First name, last name, username and user id of the sender of the current inline query.
-    * ```${iQUERY[ID]}```: Inline query ID
-    * ```${iQUERY[USER_ID]}```: User's id
-    * ```${iQUERY[FIRST_NAME]}```: User's first name
-    * ```${iQUERY[LAST_NAME]}```: User's last name
 
 ## Usage of bashbot functions
 
 #### sending messages
-To send messages use the ```send_xxx_message``` functions.
+To send messages use the `send_xxx_message` functions.
+To insert line brakes in a message place `\n` in the text. 
 
 To send regular text without any markdown use:
 ```bash
-send_text_message "${CHAT[ID]}" "lol"
+send_normal_message "${CHAT[ID]}" "lol"
 ```
 To send text with markdown:
 ```bash
@@ -223,7 +331,7 @@ To send text with html:
 send_html_message "${CHAT[ID]}" "lol <b>bold</b>"
 ```
 
-To forward messages use the ```forward``` function:
+To forward messages use the `forward` function:
 ```bash
 forward "${CHAT[ID]}" "from_chat_id" "message_id"
 ```
@@ -256,24 +364,25 @@ send_message "${CHAT[ID]}" "lol" "safe"
 
 
 #### Send files, locations, keyboards.
-To send images, videos, voice files, photos etc. use the ```send_photo``` function (remember to change the safety Regex @ line 14 of command.sh to allow sending files only from certain directories):
+To send local files or URL's (photo, video, voice, sticker, documents) use the `send_file` function.
 ```bash
-send_file "${CHAT[ID]}" "/home/user/doge.jpg" "Lool"
+send_file "${CHAT[ID]}" "/home/user/dog.jpg" "Lool" "photo"
+send_file "${CHAT[ID]}" "https://images-na.ssl-images-amazon.com/images/I/81DQ0FpoSNL._AC_SL1500_.jpg"
 ```
-To send custom keyboards use the ```send_keyboard``` function:
+To send custom keyboards use the `send_keyboard` function:
 ```bash
 send_keyboard "${CHAT[ID]}" "Text that will appear in chat?" '[ "Yep" , "No" ]' # note the single quotes!
 send_keyboard "${CHAT[ID]}" "Text that will appear in chat?" "[ \\"Yep\\" , \\"No\\" ]" # within double quotes you must escape the inside double quots
 ```
-To send locations use the ```send_location``` function:
+To send locations use the `send_location` function:
 ```bash
 send_location "${CHAT[ID]}" "Latitude" "Longitude"
 ```
-To send venues use the ```send_venue``` function:
+To send venues use the `send_venue` function:
 ```bash
 send_venue "${CHAT[ID]}" "Latitude" "Longitude" "Title" "Address" "optional foursquare id"
 ```
-To send a chat action use the ```send_action``` function.
+To send a chat action use the `send_action` function.
 Allowed values: typing for text messages, upload_photo for photos, record_video or upload_video for videos, record_audio or upload_audio for audio files, upload_document for general files, find_location for locations.
 ```bash
 send_action "${CHAT[ID]}" "action"
@@ -283,5 +392,5 @@ send_action "${CHAT[ID]}" "action"
 #### [Prev Create Bot](1_firstbot.md)
 #### [Next Advanced Usage](3_advanced.md)
 
-#### $$VERSION$$ v1.1-0-gc0eb399
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/3_advanced.md

@@ -2,7 +2,7 @@
 ## Advanced Features
 
 ### Access control
-Bashbot offers functions to check what Telegram capabilities like 'chat admin' or 'chat creator' the given user has:
+Bashbot offers functions to check what Telegram capabilities like `chat admin` or `chat creator` the given user has:
 
 ```bash
 # return true if user is admin/owner of the bot
@@ -21,7 +21,7 @@ user_is_admin "${CHAT[ID]}" "${USER[ID]}" && send_markdown_message "${CHAT[ID]}"
 
 ```
 
-In addition you can check individual capabilities of users as you must define in the file ```./botacl```:
+In addition you can check individual capabilities of users as you must define in the file `./botacl`:
 
 ```bash
 # file: botacl
@@ -54,7 +54,7 @@ ALL:*:*
 
 
 ```
-You must use the function ```user_is_allowed``` to check if a user has the capability to do something. Example: Check if user has capability to start bot.
+You must use the function `user_is_allowed` to check if a user has the capability to do something. Example: Check if user has capability to start bot.
 
 ```bash
 	case "$MESSAGE" in
@@ -62,6 +62,7 @@ You must use the function ```user_is_allowed``` to check if a user has the capab
 		# GLOBAL commands start here, only edit messages
 		'/start'*)
 			user_is_botadmin "${USER[ID]}" && send_markdown_message "${CHAT[ID]}" "You are *BOTADMIN*."
+			#  true if: user is botadmin, user is group admin, user is allowed 
 			if user_is_allowed "${USER[ID]}" "start" "${CHAT[ID]}" ; then
 				bot_help "${CHAT[ID]}"
 			else
@@ -72,10 +73,22 @@ You must use the function ```user_is_allowed``` to check if a user has the capab
 **See also [Bashbot User Access Control functions](6_reference.md#User-Access-Control)**
 
 ### Interactive Chats
-Interactive chats are short running scripts, reading user input and echo data to the user.
+Interactive chats are simple Bash scripts, reading user input as TEXT and output TEXT to the user.
 
-To create a new interactive chat script copy 'scripts/interactive.sh.clean' to e.g. 'scripts/mynewinteractive.sh', make it executable
-and then use 'start_proc' function from your bot, it's possible to pass two arguments. You find more examples for interactive scripts in 'examples'
+To create a new interactive chat script copy `scripts/interactive.sh.clean` to e.g. `scripts/mynewinteractive.sh`, make it executable
+and then use `start_proc` function from your bot, it's possible to pass two arguments. You find more examples for interactive scripts in 'examples'
+
+**Important**: The script runs standalone in a [pipeline](https://www.geeksforgeeks.org/piping-in-unix-or-linux/), it's **not possible** to use bashbot functions and variables!
+
+```bash
+commands.sh send user input as TEXT   |   script reads TEXT  |   bashbot.sh recieve output as formated TEXT (see below) 
+```
+
+In case you want to process any other data then message TEXT, I recommend to switch to a more advanced bot framework.
+In Bashbot you have to implement methods yourself to pass additional information in the message TEXT to the script.
+(only recommended for advanced bash experts).
+
+#### Example script
 
 *usage*: start_proc chat_id script arg1 arg2
 
@@ -83,7 +96,8 @@ and then use 'start_proc' function from your bot, it's possible to pass two argu
 
 *usage*: check_prog chat_id
 
-**Note:** From Version 0.96 on scripts must read user input from '$3' instead of stdin!
+
+**Note:** Scripts must read user input from '$3' instead of stdin!
 
 ```bash
 #!/bin/bash
@@ -105,8 +119,8 @@ echo -e "Your Message: ${test}\nbye!"
 
 #### message formatting and keyboards
 
-The output of the script will be processed by 'send_messages', so you can not only send text, but also keyboards, files, locations and more.
-Each newline in the output will start an new message to the user, to insert line breaks in your message you must insert ' mynewlinestartshere ' instead of a newline..
+The output of the script will be processed by `send_messages`, so you can not only send text, but also keyboards, files, locations and more.
+Each newline in the output will start an new message to the user. To have line breaks in your message you must insert `\n` instead.
 
 To open up a keyboard in an interactive script, print out the keyboard layout in the following way:
 ```bash
@@ -114,8 +128,11 @@ echo "Text that will appear in chat? mykeyboardstartshere [ \"Yep, sure\" , \"No
 ```
 Same goes for files:
 ```bash
-echo "Text that will appear in chat? myfilelocationstartshere /home/user/doge.jpg"
+echo "Text that will appear in chat? myfilestartshere /home/user/dog.jpg"
 ```
+*Note*: Use an _absolute path name_ (starting with `/`), a relative path name is relative to `data-bot-bash/upload`!
+See [send_file documentation](6_reference.md#send_file) for more information.
+	
 And buttons:
 ```bash
 echo "Text that will appear in chat. mybtextstartshere Click me myburlstartshere https://dealz.rrr.de"
@@ -130,17 +147,17 @@ echo "Text that will appear in chat. mylatstartshere 45 mylongstartshere 45 myti
 ```
 You can combine them:
 ```bash
-echo "Text that will appear in chat? mykeyboardstartshere [ \"Yep, sure\" , \"No, highly unlikely\" ] myfilelocationstartshere /home/user/doge.jpg mylatstartshere 45 mylongstartshere 45"
+echo "Text that will appear in chat? mykeyboardstartshere [ \"Yep, sure\" , \"No, highly unlikely\" ] myfilestartshere /home/user/doge.jpg mylatstartshere 45 mylongstartshere 45"
 ```
 Please note that you can either send a location or a venue, not both. To send a venue add the mytitlestartshere and the myaddressstartshere keywords.
 
-New in v0.6: To insert a linebreak in your message you can insert ```mynewlinestartshere``` in your echo command:
+To insert a line break in your message you can insert `\n` in your echo command:
 ```bash
-echo "Text that will appear in one message  mynewlinestartshere  with this text on a new line"
+echo "Text that will appear in one message \nwith this text on a new line"
 ```
 
-New in v0.7: In case you must extend a message already containing a location, a file, a keyboard etc.,
-with additionial text simply add ``` mytextstartshere additional text``` at the end of the string:
+In case you want extend a message already containing a location, a file, a keyboard etc.,
+with an additionial text simply add ` mytextstartshere additional text` at the end of the string:
 ```bash
 out="Text that will appear mylatstartshere 45 mylongstartshere 45"
 [[ "$out" != *'in chat'* ]] &&  out="$out mytextstartshere in chat."
@@ -168,7 +185,7 @@ All output of the script will be sent to the user, to stop a background job use:
 ```bash
 kill_back "${CHAT[ID]}" "jobname"
 ```
-You can also suspend and resume currently running background jobs from outside bashbot, e.g. in your startup schripts:
+You can also suspend and resume currently running background jobs from outside bashbot, e.g. in your startup scripts:
 ```bash
 ./bashbot.sh suspendback
 ./bashbot.sh resumeback
@@ -179,14 +196,14 @@ If you want to kill all background jobs permanently run:
 ./bashbot.sh killback
 
 ```
-Note: Background jobs run independent from main bot and continue running until your script exits or you stop it. Background jobs will continue running if your Bot is stopped and must be terminated separately e.g. by ```bashbot.sh killback``` 
+Note: Background jobs run independent from main bot and continue running until your script exits or you stop it. Background jobs will continue running if your Bot is stopped and must be terminated separately e.g. by `bashbot.sh killback`
 
 ### Inline queries
 **Inline queries** allow users to send commands to your bot from every chat without going to a private chat. An inline query is started if the user type the bots name, e.g. @myBot. Everything after @myBot is immediately send to the bot.
 
 In order to enable **inline mode**, send `/setinline` command to [@BotFather](https://telegram.me/botfather) and provide the placeholder text that the user will see in the input field after typing your bot’s name.
 
-The following commands allows you to send ansers to *inline queries*. To enable bashbot to process inline queries set ```INLINE="1"``` in 'mycommands.sh'.
+The following commands allows you to send ansers to *inline queries*. To enable bashbot to process inline queries set `INLINE="1"` in `mycommands.sh`.
 
 To send messages or links through an *inline query*:
 ```bash
@@ -227,7 +244,7 @@ answer_inline_query "${iQUERY[ID]}" "cached_sticker" "identifier for the sticker
 See also [answer_inline_multi, answer_inline_compose](6_reference.md#answer_inline_multi) and [mycommands.sh](../mycommands.sh) for more information.
 
 
-### Handle send message errors
+### Send message results
 
 Our examples usually do not care about errors happening while sending a message, this is OK as long your bot does not send an
 massive aoumnt of messages. By default bashbot detects if a message is not sent and try to recover when possible,
@@ -242,18 +259,19 @@ is received.
 **Note**: the values of the variables contains always the result of the LAST transmission to telegram,
 every send action will overwrite them!
 
-* ```$BOTSENT```: This array contains the parsed results from the last transmission to telegram.
-    * ```${BOTSENT[OK]}```: contains the string ```true```: after a successful transmission
-    * ```${BOTSENT[ERROR]}```: Error code if an error occurred
-    * ```${BOTSENT[DESC]}```: Description text for error
-    * ```${BOTSENT[RETRY]}```: Seconds to wait if telegram requests throtteling.
-* ```$res```: temporary variable containing the full transmission result, may be overwritten by any bashbot function.
+* `$BOTSENT`: This array contains the parsed results from the last transmission to telegram.
+    * `${BOTSENT[OK]}`: contains the string `true`: after a successful transmission
+    * `${BOTSENT[ID]}`: Message ID if OK is true
+    * `${BOTSENT[ERROR]}`: Error code if an error occurred
+    * `${BOTSENT[DESC]}`: Description text for error
+    * `${BOTSENT[RETRY]}`: Seconds to wait if telegram requests throtteling.
+* `$res`: temporary variable containing the full transmission result, may be overwritten by any bashbot function.
 
 By default you don't have to care about retry, as bashbot resend the message after the requested time automatically.
 Only if the retry fails also an error is returned. The downside is that send_message functions will wait until resend is done.
 
 If you want to disable automatic error processing  and handle all errors manually (or don't care)
-set ```BASHBOT_RETRY``` to any no zero value.
+set `BASHBOT_RETRY` to any no zero value.
 
 [Telegram Bot API error codes](https://github.com/TelegramBotAPI/errors)
 
@@ -261,9 +279,9 @@ set ```BASHBOT_RETRY``` to any no zero value.
 #### Detect bot blocked
 
 If the we can't connect to telegram, e.g. blocked from telegram server but also any other reason,
-bashbot set ```BOTSENT[ERROR]``` to ```999```.
+bashbot set `BOTSENT[ERROR]` to `999`.
 
-To get a notification on every connection problem create a function named ```bashbotBlockRecover``` and handle blocks there.
+To get a notification on every connection problem create a function named `bashbotBlockRecover` and handle blocks there.
 If the function returns true (0 or no value) bashbot will retry once and then return to the calling function.
 In case you return any non 0 value bashbot will return to the calling function without retry.
 
@@ -297,5 +315,5 @@ Note: If you disable automatic retry, se above, you disable also connection prob
 #### [Prev Getting started](2_usage.md)
 #### [Next Expert Use](4_expert.md)
 
-#### $$VERSION$$ v1.2-dev-14-g6ec00d4
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/4_expert.md

@@ -5,17 +5,17 @@
 UTF-8 is a variable length encoding of Unicode. UTF-8 is recommended as the default encoding in JSON, XML and HTML, also Telegram make use of it.
 
 The first 128 characters are regular ASCII, so it's a superset of and compatible with ASCII environments. The next 1,920 characters need
-two bytes for encoding and covers almost all ```Latin``` alphabets, also ```Greek```, ```Cyrillic```,
-```Hebrew```, ```Arabic``` and more. See [Wikipedia](https://en.wikipedia.org/wiki/UTF-8) for more details.
+two bytes for encoding and covers almost all `Latin` alphabets, also `Greek`, `Cyrillic`,
+`Hebrew`, `Arabic` and more. See [Wikipedia](https://en.wikipedia.org/wiki/UTF-8) for more details.
 
 #### Setting up your Environment
-In general ```bash``` and ```GNU``` utitities are UTF-8 aware if you to setup your environment
-and your scripts accordingly:
+In general `bash` and `GNU` utitities are UTF-8 aware if you to setup your environment
+and your scripts accordingly (_locale setting_):
 
 1. Your Terminal and Editor must support UTF-8:
-   Set Terminal and Editor locale to UTF-8, eg. in ```Settings/Configuration``` select UTF-8 (Unicode) as Charset.
+   Set Terminal and Editor locale to UTF-8, eg. in `Settings/Configuration` select UTF-8 (Unicode) as Charset.
 
-2. Set ```Shell``` environment to UTF-8 in your  ```.profile``` and your scripts. The usual settings are:
+2. Set `Shell` locale environment to UTF-8 in your  `.profile` and your scripts. The usual settings are:
 
 ```bash
 export 'LC_ALL=C.UTF-8'
@@ -31,32 +31,88 @@ export 'LANGUAGE=de_DE.UTF-8'
 ```bash
 export 'LC_ALL=en_US.UTF-8'
 export 'LANG=de_en_US.UTF-8'
-export 'LANGUAGE=den_US.UTF-8'
+export 'LANGUAGE=en_US.UTF-8'
 ```
-3. make sure your bot scripts use the correct  settings, eg. include the lines above at the beginning of your scripts
+3. make sure your bot scripts use the correct settings, eg. include the lines above at the beginning of your scripts
 
-To display all available locales on your system run ```locale -a | more```. [Gentoo Wiki](https://wiki.gentoo.org/wiki/UTF-8)
+
+#### Known locale pitfalls
+
+##### Missing C locale
+
+Even required by POSIX standard some systems (e.g. Manjaro Linux) has `C` and `C.UTF-8` locale not installed.
+If bashbot display a warning about missing locale you must install `C` and `C.UTF-8` locale.
+
+If you don't know what locales are installed on your sytsem use `locale -a` to display them.
+[Gentoo Wiki](https://wiki.gentoo.org/wiki/UTF-8).
+
+
+##### Character classes
+
+In ASCII times it was clear `[:lower:]` and `[a-z]` means ONLY the lowercase letters `[abcd...xyz]`.
+With the introduction of locales, character classes and ranges now contain all characters fitting the class definition.
+
+This means with a Latin UTF-8 locale `[:lower:]` and `[a-z]` contains also e.g. `á ø ü` etc,
+see [Unicode Latin lowercase letters](https://www.fileformat.info/info/unicode/category/Ll/list.htm)
+
+If that's ok for your script you're fine, but many scripts rely on the idea of ASCII ranges and may produce undesired results.
+
+```bash
+# try with different locales ...
+# new bash to not change your current locale!
+bash
+lower="abcö"
+
+echo "$LC_ALL $LC_COLLATE"
+[[ "$lower" =~ ^[a-z]+$ ]] && echo "Ups, $lower is all lower case!" || echo "OK, not lower case"
+
+LC_ALL="en_US.UTF-8"
+[[ "$lower" =~ ^[a-z]+$ ]] && echo "Ups, $lower is all lower case!" || echo "OK, not lower case"
+
+LC_ALL="C"
+[[ "$lower" =~ ^[a-z]+$ ]] && echo "Ups, $lower is all lower case!" || echo "OK, not lower case"
+```
+
+There are three solutions:
+
+1. list exactly the characters you want: `[abcd...]`
+2. instruct bash to use `C` locale for ranges: `shopt -s "globasciiranges"`
+3. use `LC_COLLATE` to change behavior of all programs: `export LC_COLLATE=C`
+
+
+To work independent of language and bash settings bashbot uses solution 1.: Own "ranges" if an exact match is mandatory:
+
+```bash
+azazaz='abcdefghijklmnopqrstuvwxyz'	# a-z   :lower:
+AZAZAZ='ABCDEFGHIJKLMNOPQRSTUVWXYZ'	# A-Z   :upper:
+o9o9o9='0123456789'			# 0-9   :digit:
+azAZaz="${azazaz}${AZAZAZ}"	# a-zA-Z	:alpha:
+azAZo9="${azAZaz}${o9o9o9}"	# a-zA-z0-9	:alnum:
+
+# e.g. characters allowed for key in key/value pairs
+JSSH_KEYOK="[-${azAZo9},._]"
+```
 
 #### Bashbot UTF-8 Support
 Bashbot handles all messages transparently, regardless of the charset in use. The only exception is when converting from JSON data to strings.
 
-Telegram use JSON to send / receive data. JSON encodes strings as follow: Characters not ASCII *(>127)* are escaped as sequences of ```\uxxxx``` to be regular ASCII. In addition multibyte characters, *e.g. Emoticons or Arabic characters*, are send in double byte UTF-16 notation.
-The Emoticons ``` 😁 😘 ❤️ 😊 👍 ``` are encoded as: ``` \uD83D\uDE01 \uD83D\uDE18 \u2764\uFE0F \uD83D\uDE0A \uD83D\uDC4D ```
+Telegram use JSON to send / receive data. JSON encodes strings as follow: Characters not ASCII *(>127)* are escaped as sequences of `\uxxxx` to be regular ASCII. In addition multibyte characters, *e.g. Emoticons or Arabic characters*, are send in double byte UTF-16 notation.
+The Emoticons ` 😁 😘 ❤️ 😊 👍 ` are encoded as: ` \uD83D\uDE01 \uD83D\uDE18 \u2764\uFE0F \uD83D\uDE0A \uD83D\uDC4D `
 
-**This "mixed" JSON encoding needs special handling and can not decoded from** ```echo -e``` or ```printf '%s\\n'```
+**This "mixed" JSON encoding needs special handling and can not decoded from** `echo -e` or `printf '%s\\n'`
 
-Bbashbot uses an internal, pure bash implementation which is well tested now, even there may some corner cases*.
+Bashbot uses an internal, pure bash implementation which is well tested now, even there may some corner cases*.
 
 
 ### Run as other user or system service
-Bashbot is desingned to run manually by the user who installed it. Nevertheless it's possible to run it by an other user-ID, as a system service or scheduled from cron. This is recommended if you want to bashbot run as a service.
+Bashbot is designed to run manually by the user who installed it. Nevertheless it's possible to run it by an other user-ID, as a system service or scheduled from cron. This is recommended if you want to bashbot run as a service.
 
 Setup the environment for the user you want to run bashbot and enter desired username, e.g. nobody :
 ```bash
 sudo ./bashbot.sh init
 ```
 
-Edit the file ```bashbot.rc``` and change the following lines to fit your configuration:
+Edit the file `bashbot.rc` and change the following lines to fit your configuration:
 ```bash
 #######################
 # Configuration Section
@@ -79,7 +135,7 @@ From now on use 'bashbot.rc' to manage your bot:
 ```bash
 sudo ./bashbot.rc start
 ```
-Type ```ps -ef | grep bashbot``` to verify your Bot is running as the desired user.
+Type `ps -ef | grep bashbot` to verify your Bot is running as the desired user.
 
 If your  Bot is started by 'bashbot.rc', you must use 'bashbot.rc' also to manage your Bot! The following commands are available:
 ```bash
@@ -90,16 +146,17 @@ sudo ./bashbot.rc suspendback
 sudo ./bashbot.rc resumeback
 sudo ./bashbot.rc killback
 ```
-To change back the environment to your user-ID run ```sudo ./bashbot.sh init``` again and enter your user name.
+To change back the environment to your user-ID run `sudo ./bashbot.sh init` again and enter your user name.
 
-To use bashbot as a system service include a working ```bashbot.rc``` in your init system (systemd, /etc/init.d).
+To use bashbot as a system service include a working `bashbot.rc` in your init system (systemd, /etc/init.d).
 
 ### Schedule bashbot from Cron
-An example crontab is provided in ```examples/bashbot.cron```.
+An example crontab is provided in `examples/bashbot.cron`.
 
-- If you are running bashbot with your user-ID, copy the examples lines to your crontab and remove username ```nobody```.
-- if you run bashbot as an other user or a system service edit ```examples/bashbot.cron``` to fit your needs and replace username```nobody``` with the username you want to run bashbot. copy the modified file to ```/etc/cron.d/bashbot```
+- If you are running bashbot with your user-ID, copy the examples lines to your crontab and remove username `nobody`.
+- if you run bashbot as an other user or a system service edit `examples/bashbot.cron` to fit your needs and replace username `nobody` with the username you want to run bashbot. Insert the modified lines to systm crontab file or copy to `/etc/cron.d/bashbot`
 
+**Warning:** You must use `contab -e` command to modify sytem and user crontabs, see [Cron How-To](https://help.ubuntu.com/community/CronHowto) 
 
 ### Use bashbot from CLI and scripts
 You can use bashbot to send *messages*, *locations*, *venues*, *pictures* etc. from command line and scripts
@@ -175,10 +232,10 @@ User or Chat you are in. See [Send Messages](2_usage.md#sending-messages).
 
 *Examples:* You can test this by sending messages to yourself:
 ```bash
-# fist Hello World
+# first Hello World
 send_normal_message "$(getConfigKey "botadmin")"  "Hello World! This is my first message"
 
-# now with some markdown and  HTML
+# now with some markdown and HTML
 send_markdown_message 	"$(getConfigKey "botadmin")"  '*Hello World!* _This is my first markdown message_'
 send_html_message	"$(getConfigKey "botadmin")"  '<b>Hello World!</b> <em>This is my first HTML message</em>'
 send_keyboard "$(getConfigKey "botadmin")"  'Do you like it?' '[ "Yep" , "No" ]'
@@ -202,19 +259,19 @@ This section describe how you can customize bashbot to your needs by setting env
 
 
 #### Change file locations
-In standard setup bashbot is self containing, this means you can place 'telegram-bot-bash'  any location
+In standard setup bashbot is self containing, this means you can place 'telegram-bot-bash' any location
 and run it from there. All files - program, config, data etc - will reside in 'telegram-bot-bash'.
 
-If you want to have other locations for config, data etc,  define and export the following environment variables.
+If you want to have other locations for config, data etc, define and export the following environment variables.
 **Note: all specified directories and files must exist or running 'bashbot.sh' will fail.**
 
 ##### BASHBOT_ETC
-Location of the files ```commands.sh```, ```mycommands.sh```, ```botconfig.jssh```, ```botacl``` ...
+Location of the files `commands.sh`, `mycommands.sh`, `botconfig.jssh`, `botacl` ...
 ```bash
   unset  BASHBOT_ETC     # keep in telegram-bot-bash (default)
   export BASHBOT_ETC ""  # keep in telegram-bot-bash
 
-  export BASHBOT_ETC "/etc/bashbot"  # unix like config location
+  export BASHBOT_ETC "/etc/bashbot"  # Unix-like config location
 
   export BASHBOT_ETC "/etc/bashbot/bot1"  # multibot configuration bot 1
   export BASHBOT_ETC "/etc/bashbot/bot2"  # multibot configuration bot 2
@@ -223,12 +280,12 @@ Location of the files ```commands.sh```, ```mycommands.sh```, ```botconfig.jssh`
  e.g. /etc/bashbot
 
 ##### BASHBOT_VAR
-Location of runtime data ```data-bot-bash```, ```count.jssh``` 
+Location of runtime data `data-bot-bash`, `count.jssh` 
 ```bash
   unset  BASHBOT_VAR     # keep in telegram-bot-bash (default)
   export BASHBOT_VAR ""  # keep in telegram-bot-bash
 
-  export BASHBOT_VAR "/var/spool/bashbot"  # unix like config location
+  export BASHBOT_VAR "/var/spool/bashbot"  # Unix-like config location
 
   export BASHBOT_VAR "/var/spool/bashbot/bot1"  # multibot configuration bot 1
   export BASHBOT_VAR "/var/spool/bashbot/bot2"  # multibot configuration bot 2
@@ -253,7 +310,7 @@ This is also useful if you want to force bashbot to always use full pathnames in
   unset  BASHBOT_HOME     # autodetection (default)
   export BASHBOT_HOME ""  # autodetection
 
-  export BASHBOT_HOME "/usr/local/telegram-bot-bash"	# unix like location
+  export BASHBOT_HOME "/usr/local/telegram-bot-bash"	# Unix-like location
   export BASHBOT_HOME "/usr/local/bin"	# Note: you MUST set ETC, VAR and JSONSH to other locations to make this work!
 ```
 
@@ -262,7 +319,7 @@ This is also useful if you want to force bashbot to always use full pathnames in
 #### Change config values
 
 ##### BASHBOT_URL
-Uses given URL instead of official telegram API URL, useful if you have your own telegram server or for testing.
+Uses given URL instead of official Telegram API URL, useful if you have your own telegram server or for testing.
 
 ```bash
   unset  BASHBOT_URL       # use Telegram URL https://api.telegram.org/bot<token> (default)
@@ -296,7 +353,7 @@ set BASHBOT_CURL to point to it.
 ```
 
 ##### BASHBOT_WGET
-Bashbot uses ```curl``` to communicate with telegram server. if ```curl``` is not available ```wget``` is used.
+Bashbot uses `curl` to communicate with telegram server. if `curl` is not available `wget` is used.
 If 'BASHBOT_WGET' is set to any value (not undefined or not empty) wget is used even is curl is available.  
 ```bash
   unset  BASHBOT_WGET       # use curl (default)
@@ -323,7 +380,7 @@ BASHBOT_TIMEOUT to a numeric value between 1 and 999. Any non numeric or negativ
 
 ##### BASHBOT_SLEEP
 Instead of polling permanently or with a fixed delay, bashbot offers a simple adaptive polling.
-If messages are received bashbot polls with no dealy. If no messages are available bashbot add 100ms delay
+If messages are received bashbot polls with no delay. If no messages are available bashbot add 100ms delay
 for every poll until the maximum of BASHBOT_SLEEP ms.
 ```bash
   unset  BASHBOT_SLEEP       # 5000ms (default)
@@ -378,5 +435,5 @@ for every poll until the maximum of BASHBOT_SLEEP ms.
 #### [Prev Advanced Use](3_advanced.md)
 #### [Next Best Practice](5_practice.md)
 
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/5_practice.md

@@ -10,9 +10,9 @@ In addition you should know about [BotFather, the one bot to rule them all](http
 If you don't have a github account, it may time to [setup a free account now](https://github.com/pricing)
 
 ### Add commands to mycommands.sh only
-Do not change ```bashbot.sh``` and ```commands.sh```, instead place your commands in to  ```mycommands.sh```.
-To start with a clean/minimal bot copy ```mycommands.sh.clean``` to ```mycommands.sh``` and start editing
-the message strings and place commands in the```case ... esac``` block of the function mycommands():
+Do not change `bashbot.sh` and `commands.sh`, instead place your commands in to  `mycommands.sh`.
+To start with a clean/minimal bot copy `mycommands.sh.clean` to `mycommands.sh` and start editing
+the message strings and place commands in the` case ... esac` block of the function mycommands():
 ```bash
 # file: mycommands.sh
 # your additional bashbot commands
@@ -39,13 +39,13 @@ mycommands() {
 
 ### DIsable, replace and extend global commands
 
-Global bashbot command processing, e.g. /start, /info etc. is disabled if you return a non zero value from ```mycommands.sh```,
+Global bashbot command processing, e.g. /start, /info etc. is disabled if you return a non zero value from `mycommands.sh`,
 see /start example below.
 
-To replace a global bashbot command add the same command to ```mycommands.sh``` and place ```return 1``` at the end of
+To replace a global bashbot command add the same command to `mycommands.sh` and place `return 1` at the end of
 the case block, see /kickme example below.
 
-If a command is available as a global command and in ```mycommands.sh```, plus you return a zero value (nothing or 0)
+If a command is available as a global command and in `mycommands.sh`, plus you return a zero value (nothing or 0)
 both command sections are processed. Thus you can extend global commands with additional actions, see /info example below
 
 **Learn more about [Bot commands](https://core.telegram.org/bots#commands).**
@@ -75,7 +75,7 @@ both command sections are processed. Thus you can extend global commands with ad
 
 ### Separate logic from commands
 
-If a command need more than 2-3 lines of code, you should use a function to separate logic from command. Place your functions in ```mycommands.sh``` and call the from your command. Example:
+If a command need more than 2-3 lines of code, you should use a function to separate logic from command. Place your functions in `mycommands.sh` and call the from your command. Example:
 ```bash
 # file: mycommands.sh
 # your additional bashbot commands
@@ -133,7 +133,9 @@ Line 17:
                       ^-- SC2116: Useless echo? Instead of 'cmd $(echo foo)', just use 'cmd foo'.
  
 ```
-As you can see my ```mybotcommands.inc.sh``` contains an useless echo command in 'TEXT=' assignment and can be replaced by ```TEXT="${TEXT}${WORD}"```
+
+As you can see my `mybotcommands.inc.sh` contains an useless echo command in 'TEXT=' assignment and can be replaced by `TEXT="${TEXT}${WORD}"`
+
 ```bash
 $ shellcheck -x examples/notify
 OK
@@ -158,5 +160,5 @@ The second warning is about an unused variable, this is true because in our exam
 #### [Prev Best Practice](5_practice.md)
 #### [Next Functions Reference](6_reference.md)
 
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/7_develop.md

@@ -1,44 +1,45 @@
 #### [Home](../README.md)
 
 ## Notes for bashbot developers
-This section is about help and best practices for new bashbot developers. The main focus on is creating new versions of bashbot, modules and addons, not on develop your individual bot. Nevertheless the information provided here should help your bot development also.
+This section is about help and best practices for new bashbot developers. The main focus on is creating new versions of bashbot, modules and addons, not on developing your individual bot. Nevertheless the information provided here should also help you with your bot development.
 
 If you want to provide fixes or new features [fork bashbot on github](https://help.github.com/en/articles/fork-a-repo) and provide changes as [pull request on github](https://help.github.com/en/articles/creating-a-pull-request).
 
 ### Debugging Bashbot
 Usually all bashbot output is discarded.
-If you want to get error messages (and more) start bashbot  ```./bashbot.sh startbot debug```.
+If you want to get error messages (and more) start bashbot  `./bashbot.sh startbot debug`.
 you can the change the level of verbosity of the debug argument: 
 
 ```
-	"debug"		all output is redirected to "DEBUG.log", in addition every incoming message is logged in "MESSAGE.log" and "INLINE.log"
-	"xdebug"	same as debug plus set bash option '-x' to log any executed command in "DEBUG.log"
-
-	use the command tail to watch your bot live, e.g. "tail -f DEBUG.log", to obtain more information place set -x; set +x in your code.
+	"debug"		all output is redirected to `DEBUG.log`, in addition every incoming message is logged in `MESSAGE.log` and `INLINE.log`
+	"xdebug"	same as debug plus set bash option '-x' to log any executed command in `DEBUG.log`
 ```
 
-```
-	sometimes its useful to watch the bot live in the terminal:
+Use the command `tail` to watch your bot live, e.g. `tail -f DEBUG.log`. To obtain more information place `set -x; ... set +x` around suspected code.
 
+Sometimes it's useful to watch the bot live in the terminal:
+
+```
 	"debugx"	debug output and errors are sent to terminal
-        "xdebugx"       same as debugx plus set bash option '-x' to show any executed command
-
+	"xdebugx"	same as debugx plus set bash option '-x' to show any executed command
 ```
 
+Logging of Telegram update poll is disabled by default, also in `debug` mode. To enable it without using verbose `xdebug` mode
+set `BASHBOT_UPDATELOG` to an empty value (not unset) `export BASHBOT_UPDATELOG=""`
 
 ### Modules and Addons
-**Modules** resides in ```modules/*.sh``` and are colletions of optional bashbot functions grouped by functionality. Main reason for creating modules was
+**Modules** resides in `modules/*.sh` and are colletions of optional bashbot functions grouped by functionality. Main reason for creating modules was
 to keep 'bashbot.sh' small, while extending functionality. In addition not every function is needed by all bots, so you can
 disable modules, e.g. by rename the respective module file to 'module.sh.off'.
 
 Modules must use only functions provided by 'bashbot.sh' or the module itself and should not depend on other modules or addons.
 The only mandatory module is 'module/sendMessage.sh'.
 
-If a not mandatory module is used in 'bashbot.sh' or 'commands.sh', the use of ```_is_function``` or
-```_execute_if_function``` is mandatory to catch absence of the module.
+If an optional module is used in 'bashbot.sh' or 'commands.sh', the use of `_is_function` or
+`_execute_if_function` is mandatory to catch absence of the module.
 
-**Addons** resides in ```addons/*.sh.dist``` and are not enabled by default. To activate an addon rename it to end with '.sh', e.g. by
-```cp addons/example.sh.dist addons/example.sh```. 
+**Addons** resides in `addons/*.sh.dist` and are not enabled by default. To activate an addon rename it to end with '.sh', e.g. by
+`cp addons/example.sh.dist addons/example.sh`. 
 
 Addons must register themself to BASHBOT_EVENTS at startup, e.g. to call a function every time a message is received.
 Addons works similar as 'commands.sh' and 'mycommands.sh' but are much more flexible on when functions/commands are triggered.
@@ -61,18 +62,17 @@ Note: For the same reason event function MUST return immediately! Time consuming
 
 ##### SEND RECEIVE events
 
-An RECEIVE event is executed when a Message is received, same iQuery / Message variables are available as in commands.sh
+A RECEIVE event is executed when a Message is received, same iQuery / Message variables are available as in commands.sh
 
-* BASHBOT_EVENT_INLINE		an inline query is received
-
-* BASHBOT_EVENT_MESSAGE		any of the following message types is received
-    * BASHBOT_EVENT_TEXT	a message containing text is received
-    * BASHBOT_EVENT_CMD		a message containing a command is received (starts with /)
-    * BASHBOT_EVENT_REPLYTO	a reply to a message is received
-    * BASHBOT_EVENT_FORWARD	a forwarded message is received
-    * BASHBOT_EVENT_CONTACT	a contact is received
-    * BASHBOT_EVENT_LOCATION	a location or a venue is received
-    * BASHBOT_EVENT_FILE	a file is received
+* `BASHBOT_EVENT_INLINE`    an inline query is received
+* `BASHBOT_EVENT_MESSAGE`   any of the following message types is received
+* `BASHBOT_EVENT_TEXT`      a message containing text is received
+* `BASHBOT_EVENT_CMD`       a message containing a command is received (starts with /)
+* `BASHBOT_EVENT_REPLYTO`   a reply to a message is received
+* `BASHBOT_EVENT_FORWARD`   a forwarded message is received
+* `BASHBOT_EVENT_CONTACT`   a contact is received
+* `BASHBOT_EVENT_LOCATION`  a location or a venue is received
+* `BASHBOT_EVENT_FILE`      a file is received
 
 *usage*: BASHBOT_EVENT_xxx[ "unique-name" ]="callback"
 
@@ -82,6 +82,7 @@ An RECEIVE event is executed when a Message is received, same iQuery / Message v
 and "unique-name" is the name provided when registering the event. 
 
 *Example:* Register a function to echo to any Text sent to the bot
+
 ```bash
 # register callback:
 BASHBOT_EVENT_TEXT["example_1"]="example_echo"
@@ -100,7 +101,7 @@ example_echo() {
 
 An SEND event is executed when a Message is send to telegram.
 
-* BASHBOT_EVENT_SEND	is executed if data is send or uploaded to Telegram server
+* `BASHBOT_EVENT_SEND` is executed if data is send or uploaded to Telegram server
 
 In contrast to other events, BASHBOT_EVENT_SEND is executed in a sub shell, so there is no need to spawn
 a background process for longer running commands and changes to variables are not persistent!
@@ -130,13 +131,13 @@ example_log(){
 
 Important: Bashbot timer tick is disabled by default and must be enabled by setting BASHBOT_START_TIMER to any value not zero.
 
-* BASHBOT_EVENT_TIMER		executed every minute and can be used in 3 variants: oneshot, once a minute, every X minutes.
+* `BASHBOT_EVENT_TIMER` is executed every minute and can be used in 3 variants: oneshot, once a minute, every X minutes.
 
 Registering to BASHBOT_EVENT_TIMER works similar as for message events, but you must add a timing argument to the name.
 EVENT_TIMER is triggered every 60s and waits until the current running command is finished, so it's not exactly every
 minute, but once a minute.
 
-Every time EVENT_TIMER is triggered the variable "EVENT_TIMER" is increased. each callback is executed if ```EVENT_TIMER % time``` is '0' (true).
+Every time EVENT_TIMER is triggered the variable "EVENT_TIMER" is increased. each callback is executed if `EVENT_TIMER % time` is '0' (true).
 This means if you register an every 5 minutes callback first execution may < 5 Minutes, all subsequent executions are once every 5. Minute.
 
 *usage:* BASHBOT_EVENT_TIMER[ "name" , "time" ], where time is:
@@ -146,7 +147,7 @@ This means if you register an every 5 minutes callback first execution may < 5 M
     * x	execute every x minutes
     * -x execute once WITHIN the next x Minutes (next 10 Minutes since start "event")
 
-Note: If you want exact "in x minutes" use "EVENT_TIMER plus x" as time: ```-(EVENT_TIMER + x)```
+Note: If you want exact "in x minutes" use "EVENT_TIMER" as reference: `(EVENT_TIMER +x)`
 
 *Example:*
 ```bash
@@ -165,53 +166,61 @@ BASHBOT_EVENT_TIMER["example_every5","5"]="example_every5min"
 # execute once on the next 10 minutes since start "event"
 BASHBOT_EVENT_TIMER["example_10min","-10"]="example_in10min"
 
-# once in exact 10 minutes
-BASHBOT_EVENT_TIMER["example_10min","$(( (EVENT_TIMER+10) * -1 ))"]="example_in10min"
+# once in exact 10 minutes, note the -
+BASHBOT_EVENT_TIMER["example_10min","-$(( EVENT_TIMER+10 ))"]="example_in10min"
 
 ```
 
 ----
 
-#### Create a stripped down Version of your Bot
+#### Create a stripped down version of your Bot
 Currently bashbot is more a bot development environment than a bot, containing examples, developer scripts, modules, documentation and more.
 You don't need all these files after you're finished with your cool new bot.
 
 Let's create a stripped down version:
 
-- delete all modules you do not need from 'modules', e.g. 'modules/inline.sh' if you don't use inline queries
-- delete not needed standard commands and messages from 'commands.sh'
-- delete not needed commands and functions from 'mycommands.sh'
-- run ```dev/make-standalone.sh``` to create a a stripped down version of your bo
+- delete all modules you do not need from `modules`, e.g. `modules/inline.sh` if you don't use inline queries
+- delete unused standard commands and messages from `commands.sh`
+- delete unused commands and functions from `mycommands.sh`
+- run `dev/make-standalone.sh` to create a a stripped down version of your bot
 
-Now have a look at the directory 'standalone', here you find the files 'bashbot.sh' and 'commands.sh' containing everything to run your bot.
+Now have a look at the directory `standalone`, here you find the files `bashbot.sh` and `commands.sh` containing everything to run your bot.
 [Download make-standalone.sh](https://github.com/topkecleon/telegram-bot-bash/blob/master/dev/make-standalone.sh) from github.
 
 ### Setup your develop environment
 
-1. install git, install [shellcheck](5_practice.md#Test-your-Bot-with-shellcheck)
+1. install the commands git [shellcheck](5_practice.md#Test-your-Bot-with-shellcheck) bc pandoc zip codespell
 2. setup your [environment for UTF-8](4_expert.md#Setting-up-your-Environment)
-3. clone your bashbot fork to a new directory ```git clone https://github.com/<YOURNAME>/telegram-bot-bash.git```, replace ```<YOURNAME>``` with your username on github
-4. create and change to your develop branch ```git checkout -b <YOURBRANCH>```, replace ```<YOURBRANCH>``` with the name you want to name it, e.g. 'develop'
-5. give your (dev) fork a new version tag: ```git tag vx.xx```(optional) 
-6. setup github hooks by running ```dev/install-hooks.sh``` (optional)
+3. clone your bashbot fork to a new directory `git clone https://github.com/<YOURNAME>/telegram-bot-bash.git`, replace `<YOURNAME>` with your username on github
+4. create and change to your develop branch `git checkout -b develop`
+5. give your (dev) fork a new version tag: `git tag v1.xx`
+6. setup github hooks by running `dev/install-hooks.sh`
+
+Run `dev/make-distribution.sh` to create installation archives and a test installation in `DIST/`.
+To update the test installation, e.g. after git pull, local changes or switch master/develop, run `dev/make-distribution.sh` again.
+
+Note for Debian: Debian Buster ships older versions of many utilities, pls try to install from [buster-backports](https://backports.debian.org/Instructions/)
+```bash
+sudo apt-get -t buster-backports install git shellcheck pandoc codespell curl
+```
 
 #### Test, Add, Push changes
 A typical bashbot develop loop looks as follow:
 
 1. start developing - *change, copy, edit bashbot files ...*
-2. after changing a bash sript: ```shellcheck -x script.sh```
-3. ```dev/all-tests.sh``` - *in case if errors back to 2.*
-4. ```dev/git-add.sh``` - *check for changed files, update version string, run git add*
-5. ```git commit -m "COMMIT MESSAGE"; git push```
+2. after changing a bash script: `shellcheck -x script.sh`
+3. `dev/all-tests.sh` - *in case if errors back to 2.*
+4. `dev/git-add.sh` - *check for changed files, update version string, run git add*
+5. `git commit -m "COMMIT MESSAGE"; git push`
 
 
 **If you setup your dev environment with hooks and use the scripts above, versioning, adding and testing is done automatically.**
 
 #### common commands
 We state bashbot is a bash only bot, but this is not true. bashbot is a bash script using bash features PLUS external commands.
-Usually bash is used in a unix/linux environment where many (GNU) commands are available, but if commands are missing, bashbot may not work.
+Usually bash is used in a Linux/Unix environment where many (GNU) commands are available, but if commands are missing, bashbot may not work.
 
-To avoid this and make bashbot working on as many platforms as possible - from embedded linux to mainframe - I recommend to restrict
+To avoid this and make bashbot working on as many platforms as possible - from embedded Linux to mainframe - I recommend to restrict
 ourself to the common commands provided by bash and coreutils/busybox/toybox.
 See [Bash Builtins](https://www.gnu.org/software/bash/manual/html_node/Shell-Builtin-Commands.html),
 [coreutils](https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands),
@@ -243,6 +252,8 @@ data="$(cat file)" -> data="$(<"file")"
 
 DIR="$(dirname $0) -> DIR="${0%/*}"
 
+date -> printf"%(%c)T\n" -1	# 100 times faster!
+
 PROG="($basename $0)" -> PROG="${0##*/}*
 
 ADDME="$ADDME something to add" -> ADDME+=" something to add""
@@ -250,29 +261,58 @@ ADDME="$ADDME something to add" -> ADDME+=" something to add""
 VAR="$(( 1 + 2 ))" -> (( var=1+2 ))
 
 INDEX="$(( ${INDEX} + 1 ))" -> (( INDEX++ ))
-
 ```
 For more examples see [Pure bash bible](https://github.com/dylanaraps/pure-bash-bible)
 
+The special variable `$_` stores the expanded __last__ argument of the previous command.
+This allows a nice optimisation in combination with the no-op command `:`, but be aware of `$_` pitfalls.
+
+```bash
+# $_ example: mkdir plus cd to it
+mkdir "somedir-$$" && cd "$_"	# somedir-1234 (process id)
+
+# manipulate a variable multiple times without storing intermediate results
+foo="1a23_b__x###"
+...
+: "${foo//[0-9]}"	# a_b__x###
+: "${_%%#*}"		# a_b__x
+bar="${_/__x/_c}"	# a_b_c
+
+
+# BE AWARE OF ...
+# pitfall missing quotes: $_ is LAST arg
+: ${SOMEVAR}		# "String in var" $_ -> "var" 
+: $(<"file")		# "Content     of\n  file" $_ -> "file"
+
+# pitfall [ vs. test command
+[ -n "xxx" ] && echo "$_"	# $_ -> "]" 
+test -n "xxx" && echo "$_"	# $_ -> "xxx" 
+
+# pitfall command substitution: globbing and IFS is applied!
+: "$(echo "a* is born")"# $_ -> a* is globbed even quoted!
+: "$(echo "a   b    c")"# $_ -> "a b c"
+: "$(<"file")"		# "Content     of\n  file" $_ -> "Content of file"
+```
+
 #### Prepare a new version
 After some development it may time to create a new version for the users. a new version can be in sub version upgrade, e.g. for fixes and smaller additions or
-a new release version for new features. To mark a new version use ```git tag NEWVERSION``` and run ```dev/version.sh``` to update all version strings.
+a new release version for new features. To mark a new version use `git tag NEWVERSION` and run `dev/version.sh` to update all version strings.
 
 Usually I start with pre versions and when everything looks good I push out a release candidate (rc) and finally the new version.
 ```
  v0.x-devx -> v0.x-prex -> v0.x-rc -> v0.x  ... 0.x+1-dev ...
 ```
 
-If you release a new Version run ```dev/make-distribution.sh``` to create the zip and tar.gz archives in the dist directory and attach them to the github release. Do not forget to delete directory dist afterwards.
+If you release a new Version run `dev/make-distribution.sh` to create the zip and tar.gz archives in the dist directory and attach them to the github release. Do not forget to delete directory dist afterwards.
 
 #### Versioning
 
 Bashbot is tagged with version numbers. If you start a new development cycle you can tag your fork with a version higher than the current version.
-E.g. if you fork 'v0.60' the next develop version should tagged as ```git tag "v0.61-dev"``` for fixes or ```git tag "v0.70-dev"``` for new features.
+E.g. if you fork 'v0.60' the next develop version should tagged as `git tag "v0.61-dev"` for fixes or `git tag "v0.70-dev"` for new features.
 
-To get the current version name of your develepment fork run ```git describe --tags```. The output looks like ```v0.70-dev-6-g3fb7796``` where your version tag is followed by the number of commits since you tag your branch and followed by the latest commit hash. see also [comments in version.sh](../dev/version.sh)
+To get the current version name of your develepment fork run `git describe --tags`. The output looks like `v0.70-dev-6-g3fb7796` where your version tag is followed by the number of commits since you tag your branch and followed by the latest commit hash. see also [comments in version.sh](../dev/version.sh)
 
-To update the Version Number in files run ```dev/version.sh files```, it will update the line '#### $$VERSION$$ ###' in all files to the current version name.
+To update the Version Number in files run `dev/version.sh files`, it will update the line '#### $$VERSION$$ ###' in all files to the current version name.
 To update version in all files run 'dev/version.sh' without parameter.
 
 
@@ -280,27 +320,27 @@ To update version in all files run 'dev/version.sh' without parameter.
 
 For a shell script running as a service it's important to be paranoid about quoting, globbing and other common problems. So it's a must to run shellchek on all shell scripts before you commit a change. this is automated by a git hook activated in Setup step 6.
 
-To run shellcheck for a single script run ```shellcheck -x script.sh```, to check all schripts run ```dev/hooks/pre-commit.sh```.
+To run shellcheck for a single script run `shellcheck -x script.sh`, to check all scripts run `dev/hooks/pre-commit.sh`.
 
 
 ### bashbot test suite
-Starting with version 0.70 bashbot has a test suite. To start testsuite run ```dev/all-tests.sh```. all-tests.sh will return 'SUCCESS' only if all tests pass.
+Starting with version 0.70 bashbot has a test suite. To start the test suite run `dev/all-tests.sh`. all-tests.sh will return 'SUCCESS' if ALL tests pass.
 
 #### enabling / disabling tests
 
-All tests are placed in the directory  ```test```. To disable a test remove the execute flag from the '*-test.sh' script, to (re)enable a test make the script executable again.
+All tests are placed in the directory `test`. To disable a test remove the execute flag from the '*-test.sh' script, to (re)enable a test make the script executable again.
 
 
 #### creating new tests
-To create a new test run ```test/ADD-test-new.sh``` and answer the questions, it will create the usually needed files and dirs:
+To create a new test run `test/ADD-test-new.sh` and answer the questions, it will create the usually needed files and dirs:
 
-Each test consists of a script script named after ```p-name-test.sh``` *(where p is test pass 'a-z' and name the name
-of your test)* and an optional dir ```p-name-test/``` *(script name minus '.sh')* for additional files.
+Each test consists of a script script named after `p-name-test.sh` *(where p is test pass 'a-z' and name the name
+of your test)* and an optional dir `p-name-test/` *(script name minus '.sh')* for additional files.
 
 Tests with no dependency to other tests will run in pass 'a', tests which need an initialized bashbot environment must run in pass 'd' or later. 
 A temporary test environment is created when 'ALL-tests.sh' starts and deleted after all tests are finished.
 
-The file ```ALL-tests.inc.sh``` must be included from all tests and provide the test environment as shell variables:
+The file `ALL-tests.inc.sh` must be included from all tests and provide the test environment as shell variables:
 ```bash
 # Test Environment
  TESTME="$(basename "$0")"
@@ -317,7 +357,7 @@ The file ```ALL-tests.inc.sh``` must be included from all tests and provide the
  ADMINFILE="botadmin"
  DATADIR="data-bot-bash"
 
-# SUCCESS NOSUCCES -> echo "${SUCCESS}" or echo "${NOSUCCESS}" 
+# SUCCESS NOSUCCESS -> echo "${SUCCESS}" or echo "${NOSUCCESS}" 
  SUCCESS="   OK"
  NOSUCCESS="   FAILED!"
 
@@ -347,5 +387,5 @@ fi
 
 #### [Prev Function Reference](6_reference.md)
 
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 

doc/bashbot.ascii

@@ -0,0 +1,22 @@
+
+                      ..
+                     ****                     
+                ****oooooo*****             
+           *****ooooooooooooo*****        
+       *****oooooooooooooooooooooo****  
+   ****oooooooooooooooooooooooooooooooo**
+ *.*oooooooooooooooooooooooooooooooooooo**
+ *.ooooooooooooooooooooooooooooooooo**....
+ *.oooooooooooooooooooooooooooo**.........
+ *.oooooooooooooooooooooooo**.............
+ *.ooooooooooooooooooo**..................     ____               _      _             _   
+ *.ooooooooooooooooo*.......,.............    |  _ \             | |    | |           | |  
+ *.ooooooooooooooooo*.....,***,...........    | |_) |  __ _  ___ | |__  | |__    ___  | |_ 
+ *.ooooooooooooooooo*....o*...............    |  _ <  / _` |/ __|| '_ \ | '_ \  / _ \ | __|
+ *.ooooooooooooooooo*....*o***,...........    | |_) || (_| |\__ \| | | || |_) || (_) || |_ 
+ *.*oooooooooooooooo*........o*.....oo....    |____/  \__,_||___/|_| |_||_.__/  \___/  \__|
+   ****ooooooooooooo*....`***....oo.....*
+       *****oooooooo*......*..oo.....**  
+           ******ooo*.............*       
+                ***o*........**             
+                    **...**                    

examples/README.html

@@ -1,61 +0,0 @@
-<!DOCTYPE html>
-<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
-<head>
-  <meta charset="utf-8" />
-  <meta name="generator" content="pandoc" />
-  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
-  <title>Bashobot Documentation - examples/README.html</title>
-  <style>
-      code{white-space: pre-wrap;}
-      span.smallcaps{font-variant: small-caps;}
-      span.underline{text-decoration: underline;}
-      div.column{display: inline-block; vertical-align: top; width: 50%;}
-  </style>
-  <!--[if lt IE 9]>
-    <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
-  <![endif]-->
-</head>
-<body>
-<header id="title-block-header">
-<h1 class="title">Bashobot Documentation - examples/README.html</h1>
-</header>
-<h4><a href="../README.html">Home</a></h4>
-<h2>Bashbot examples</h2>
-<h3>bashbot multi</h3>
-<p>An example wrapper to run multiple instances of bashbot, use <code>./bashbot-multi.sh botname command</code></p>
-<h3>bashbot.cron</h3>
-<p>An example crontab is provided in <code>examples/bashbot.cron</code>, see <a href="../html/4_expert.html">Expert use</a></p>
-<h3>Interactive chats</h3>
-<p>Two examples for interactive scripts are provided as <strong>calc.sh</strong> and <strong>question.sh</strong>, see <a href="../html/3_advanced.html">Advanced use</a></p>
-<h3>Background scripts</h3>
-<p>Background jobs are an easy way to provide sceduled messages or alerts if something happens. <strong>notify.sh</strong> is a simple example on how to send a message every x seonds, e.g. current time.</p>
-<p><strong>background-scripts</strong> contains a more useful example on how to start and stop different scripts plus some example background scripts.</p>
-<pre><code>    mycommands.sh - /run_xxx and /kill-xxx will start any script named run_xxx.sh
-
-    run_diskusage.sh - shows disk usage every 100 seconds
-    run_filename.sh - shown the name of new files in a named dir
-    run_filecontent.sh  - shown the content of new files in a named dir
-    run_notify.sh - same as notify.sh
-</code></pre>
-<p><strong>Note:</strong> Output of system commands often contains newlines, each newline results in a telegram message, the function 'send_telegram' in mycommands.sh avoids this by converting each newline to ' mynewlinestartshere ' before output the string.</p>
-<h3>System Status</h3>
-<p><strong>send-system-status</strong> contains an example for commands showing status of different subsystems. This example is adapted from https://github.com/RG72/telegram-bot-bash to current bashbot commands, but not fully tested. This will show how easy you can convert existing bots.</p>
-<pre><code>    mycommands.sh - commands to show system status
-    botacl - controls who can show system status
-
-*Available commands*:
-    /se *sensors*
-    /smb *smbstatus*
-    /free *memory status*
-    /md *raid status*
-    /lvm *lvm status*
-    /lvsd *Datailed lvm status*
-    /df *disk space*
-    /ifconfig *ifconfig output*
-    /smart *sda* _smart status for sda drive_
-</code></pre>
-<h3>External scripts</h3>
-<p><strong>external-use</strong> will contain some examples on how to send messages from external scripts to Telegram chats or users.</p>
-<h4>$$VERSION$$ v0.98-pre2-0-ga597303</h4>
-</body>
-</html>

examples/README.md

@@ -56,6 +56,10 @@ convert existing bots.
 **jsonDB-keybords** contains a stripped down real world example from my bot showing the usage of jsonDB to store and retrieve values
 plus use of keyboards in private chats. It's an extended version of mycommands.sh.dist. Messages and help are in german.
 
-#### $$VERSION$$ v1.0-0-g99217c4
+### Webhook
+
+**Webhook** contains instructions on how use webhook API to get updates from telegram instead polling Telegram server.
+
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 

examples/background-scripts/mycommands.sh

@@ -6,12 +6,12 @@ export res
 # your additional bashbot commands ...
 mycommands() {
 
-	case "$MESSAGE" in
+	case "${MESSAGE}" in
 		'/run_'*) 
 			myback="run_${MESSAGE#*_}"
 			if [ -x "./${myback}.sh" ]; then
 			    checkback "${myback}"
-			    if [ "$res" -gt 0 ] ; then
+			    if [ "${res}" -gt 0 ] ; then
 				send_normal_message "${CHAT[ID]}" "Start ${myback}, use /kill${myback} to stop it."
 				background "./${myback}.sh" "${myback}"
 			    else
@@ -23,7 +23,7 @@ mycommands() {
 			myback="run_${MESSAGE#*_}"
 			if [ -x "./${myback}.sh" ]; then
 			    checkback "${myback}"
-			    if [ "$res" -eq 0 ] ; then
+			    if [ "${res}" -eq 0 ] ; then
 				killback "${myback}"
 				send_normal_message "${CHAT[ID]}" "Stopping ${myback}, use /run_${myback} to start again."
 			    else
@@ -56,7 +56,7 @@ watch_dir_loop() {
 		echo "$(date): new file: ${newfile}" >>"$0.log"
 		# note: loop callback must a function in the calling script! 
 		if _is_function loop_callback ; then
-			loop_callback "$1/$newfile"
+			loop_callback "$1/${newfile}"
 		else
 			echo "ERROR: loop_callback not found!" >&2
 			exit 1
@@ -94,8 +94,8 @@ output_file() {
 	sed <<< "${1}" '
 	s/ *mynewlinestartshere */\n/
 	s/ my[a-z]\{3,15}\(start\|ends\)here.*//g
-	' >"$publish$$"
-	cat  "$publish" >>"$publish$$"
+	' >"${publish}$$"
+	cat  "${publish}" >>"${publish}$$"
 	mv "${publish}$$" "${publish}"
 } # >>"$0.log" 2>&1
 

examples/background-scripts/run_diskusage.sh

@@ -1,10 +1,10 @@
 #!/bin/bash
-# file: run_diskcusage.sh
+# file: run_diskusage.sh
 # example for an background job display a system value
 #
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 ######
 # parameters
@@ -28,8 +28,8 @@ cat >/dev/null &
 source "./mycommands.sh"
 
 # check if $1 is a number
-re='^[0-9]+$'
-if [[ $1 =~ $re ]] ; then
+regex='^[0-9]+$'
+if [[ $1 =~ ${regex} ]] ; then
 	SLEEP="$1"
 else
 	SLEEP=100 # time between time notifications
@@ -37,13 +37,13 @@ fi
 
 NEWLINE=$'\n'
 
-# output disk usgae every $1 seconds
+# output disk usage every $1 seconds
 WAIT=0
-while sleep $WAIT
+while sleep "${WAIT}"
 do
 	output_telegram "Current Disk usage ${NEWLINE} $(df -h / /tmp /usr /var /home)"
 	# only for testing, delete echo line for production ...
 	echo "Current Disk usage ${NEWLINE} $(df -h / /tmp /usr /var /home)"
-	WAIT="$SLEEP"
+	WAIT="${SLEEP}"
 done
 

examples/background-scripts/run_filecontent.sh

@@ -2,7 +2,7 @@
 # file:  run_filename
 # background job to display content of all new files in WATCHDIR
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 ######
 # parameters
@@ -40,4 +40,4 @@ loop_callback() {
 	output_telegram "Contents of ${1}: ${NEWLINE} $(cat "${1}")"
 }
 
-watch_dir_loop "$WATCHDIR"
+watch_dir_loop "${WATCHDIR}"

examples/background-scripts/run_filename.sh

@@ -2,7 +2,7 @@
 # file:  run_filename
 # background job to display all new files in WATCHDIR
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 ######
 # parameters
@@ -38,5 +38,5 @@ loop_callback() {
 	echo "New file ${1} created in ${WATCHDIR}!"
 }
 
-watch_dir_loop "$WATCHDIR"
+watch_dir_loop "${WATCHDIR}"
 

examples/background-scripts/run_notify.sh

@@ -4,7 +4,7 @@
 #
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 ######
 # parameters
@@ -25,15 +25,15 @@ unset IFS
 cat >/dev/null & 
 
 # check if $1 is a number
-re='^[0-9]+$'
-if [[ $1 =~ $re ]] ; then
+regex='^[0-9]+$'
+if [[ "$1" =~ ${regex} ]] ; then
 	SLEEP="$1"
 else
 	SLEEP=10 # time between time notifications
 fi
 
 # output current time every $1 seconds
-while sleep $SLEEP
+while sleep "${SLEEP}"
 do
 	date "+* It's %k:%M:%S o' clock ..."
 done

examples/bash2env.sh

@@ -5,8 +5,8 @@
 
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
-
-#### $$VERSION$$ v1.0-0-g99217c4
+# shellcheck disable=SC1117
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # adjust your language setting here
 # https://github.com/topkecleon/telegram-bot-bash#setting-up-your-environment
@@ -19,7 +19,7 @@ MYSHEBANG=""
 
 ################
 # uncomment one of the following lines to make the conversion
-# linux/unix  bash
+# Linux/Unix  bash
 # MYSHEBANG="#!/bin/bash"
 
 # BSD bash

examples/bashbot-multi.sh

@@ -2,14 +2,14 @@
 # file. multibot.sh
 # description:  run multiple telegram bots from one installation
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 
-if [ "${2}" = "" ] || [ "${2}" = "-h" ]; then
+if [ "$2" = "" ] || [ "$2" = "-h" ]; then
 	echo "Usage: $0 botname command"
 	exit 1
 fi
 
-BOT="${1}"
+BOT="$1"
 [ "${#BOT}" -lt 5 ] && echo "Botname must have a minimum length of 5 characters" && exit 1
 
 # where should the bots live?
@@ -21,7 +21,7 @@ if true; then
   VAR="${BINDIR}"
 
 else
-  # alternative linux like locations
+  # alternative Linux-like locations
   BINDIR="/usr/local/bin"
   ETC="/etc/bashbot"
   VAR="/var/bashbot"

examples/bashbot.cron

@@ -7,7 +7,7 @@
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 
 SHELL=/bin/sh
@@ -23,13 +23,14 @@ MAILTO=root
 # │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
 # │ │ │ │ │                                   7 is also Sunday on some systems)
 # │ │ │ │ │
-# │ │ │ │ │
+# │ │ │ │ │   ┌───────────── run as user (must be omited in users crontab
+# │ │ │ │ │   |
 # * * * * *  USER command to execute
 # * * * * *  root  echo "run every minute!"
 
 # run as www every day at 0:00 plus random sleep between 0-3h
-  0 0 * * * nobody sleep "$((RANDOM \% 180 ))m" ; /usr/local/telegram-bot-bash/bashbot.sh start	# (re)start bot
-  0 0 * * * nobody sleep "$((RANDOM \% 180 ))m" ; /usr/local/telegram-bot-bash/bashbot.sh resumeback	# (re)start background jobs
+  0 0 * * * nobody sleep "$((RANDOM \% 180 ))m" ; /usr/local/telegram-bot-bash/bashbot.rc start	# (re)start bot
+  0 0 * * * nobody sleep "$((RANDOM \% 180 ))m" ; /usr/local/telegram-bot-bash/bashbot.rc resumeback	# (re)start background jobs
 
 # run as www on 24 of Dec, 12:00
   0 12 24 12 * nobody /usr/local/telegram-bot-bash/bashbot.sh broadcast "X-Mas shopping is over!"	# broadcast a message

examples/calc.sh

@@ -1,11 +1,18 @@
 #!/bin/bash
-# file: calc.sh
-# example for an interactive chat, run with startproc calc.sh
+########################################################################
+#
+# File: calc.sh
+#
+# Description: example for an background job, see mycommands.sh.dist
+#
+# Usage: runback calc  example/calc.sh  - or run in terminal
+#        killback calc - to stop background job
 #
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
+########################################################################
 
 ######
 # parameters
@@ -23,22 +30,21 @@ export 'LANGUAGE=C.UTF-8'
 unset IFS
 # set -f # if you are paranoid use set -f to disable globbing
 
-echo 'Starting Calculator ...'
-echo 'Enter first number.'
+printf 'Starting Calculator ...\n'
+printf 'Enter first number.\n'
 read -r A <"${INPUT}"
-echo 'Enter second number.'
+printf 'Enter second number.\n'
 read -r B <"${INPUT}"
-echo 'Select Operation: mykeyboardstartshere [ "Addition" , "Subtraction" , "Multiplication" , "Division" ,  "Cancel" ]'
+printf 'Select Operation: mykeyboardstartshere [ "Addition" , "Subtraction" , "Multiplication" , "Division" ,  "Cancel" ]\n'
 read -r opt <"${INPUT}"
-echo -n 'Result: '
-case $opt in
-	'add'* | 'Add'* ) res="$(( A + B ))" ;;
-	'sub'* | 'Sub'* ) res="$(( A - B ))" ;;
-	'mul'* | 'Mul'* ) res="$(( A * B ))" ;;
-	'div'* | 'Div'* ) res="$(( A / B ))" ;;
-	'can'* | 'Can'* ) res="abort!" ;;
-		* ) echo "unknown operator!";;
+printf 'Result: '
+case ${opt,,} in
+	'a'*) res="$(( A + B ))" ;;
+	's'*) res="$(( A - B ))" ;;
+	'm'*) res="$(( A * B ))" ;;
+	'd'*) res="$(( A / B ))" ;;
+	'c'*) res="abort!" ;;
+	* ) printf "unknown operator!\n";;
 esac
-echo "$res"
-echo "Bye .."
+printf "%s\nBye ...\n" "${res}"
 

examples/jsonDB-keyboard/mycommands.sh

@@ -1,19 +1,25 @@
 #!/bin/bash
-# files: mycommands.sh.dist
-# copy to mycommands.sh and add all your commands and functions here ...
+#===============================================================================
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#          FILE: mycommands.sh
+# 
+#         USAGE: will be included from commands.sh 
+# 
+#   DESCRIPTION: real world example with jsshDB and send_keyboard
+# 
+#        AUTHOR: KayM (), kay@rrr.de
+#          DATE: 19.12.2020 19:03
 #
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 # shellcheck disable=SC2154
 # shellcheck disable=SC2034
 
-# within *xxx* markup we need only one \ to escape a '_', e.g. my\_stupid\_bot (stupid V1 markup)
 bashbot_title='*Hallo, ich bin der @'"${ME//_/\\_}"'. Ich suche und finde Dealz!*'
 
-# outside *xxxx* markup we need two \\ to escape a '_', e.g. -> my\\_stupid\\_bot
 bashbot_shelp="${bashbot_title}"'
 
-Schicke mir hier -> @'"${ME//_/\\\\_}"' <- einen Befehl mit *max. 50 Begriffen*. Ein Begriff muss mindesten 4 Zeichen lang sein.
+Schicke mir https://t.me/'"${ME//_/\\_}"' einen Befehl mit *max. 50 Begriffen*. Ein Begriff muss mindesten 4 Zeichen lang sein.
 
 *Befehle zum Suchen / Tracken  von Dealz:*
   /search  /suche _Suche in Deals der letzen 48h,
@@ -22,139 +28,56 @@ Schicke mir hier -> @'"${ME//_/\\\\_}"' <- einen Befehl mit *max. 50 Begriffen*.
   /newtrack   _Tracking mit neuen Begriffen,
   z.B.:_ /newtrack    Apple  mavic.\*air
   /track /addkey   _zusätzlicher Begriff zum Tracken_
-  /addblack   _Begriff darf nicht vorkommen_
   /delkey       _lösche Begriff_
-  /delblack   _lösche Blacklist Begriff_
 
   /listtrack   _zeigt aktuell getrackte Begriffe_
   /delkey ALL  _löscht alle Begriffe_ 
   /stattrack   _Infos zum aktiven Tracking_
 
-Kurzform: /such /sea /newt /tra /addk /addb /delk /delb /lst /statt
-
-*Platzhalter in Suchbegriffen:* 
-    *.*\*  = egal/alles:  1tb*.*\*ssd
-        ⇨ 1Tb-ssd, 1tb ssd,  1Tb Supi ssd
-    *.*    = genau ein Zeichen: *.*tb 
-        ⇨ 1Tb,  5TB,  atb, ztb
-    *_*   = Wortanfang/end:  \\_Mavic\\_
-        ⇨ das Wort  _Mavic_
-
-Die Suche ignoriert Gross und Klein Schreibung, Satzzeichen, Klammern und "-", ebenso Links und Bilder. Mehr zu Regular Expressions (RegEx) http://www.regexe.de/hilfe.jsp
 '
 
-bashbot_whelp="${bashbot_title}"'
+bashbot_commands='*Meine Befehle*:
+  /info      _- Info zum Bot_
+  /suche     _- Suchen und Tracken von Dealz_
+  /hilfe    _- Hilfe zu weiteren Befehlen_
 
-Schicke mir hier -> @'"${ME//_/\\\\_}"' <- einen Befehl mit *max. 50 Artikeln (ASIN)* zum Beobachten.
-
-*Befehle zum Beobachten von Amzon Artikeln*
-
-  /newwatch   _Beobachten neuer Artikel_
-  z.B.:_ /newwatch    ASIN ASIN_
-  /watch /addwatch   _weiteren Artikel beobachten_
-  /delwatch     _löscht Artikel aus Beobachtung_
-
-  /listwatch     _zeigt aktuell beobachtete Artikel_
-  /delwatch ALL  _löscht alle Artikel_ 
-
-  /notify        _einmalig warten bis Verfügar_
-  /listnotify    _zeigt aktuell wartende Artikel_
-  /delnotify     _löscht alle wartenden Artikel_
-
-Kurzform: /neww /wat /addw /delw /listw /noti /listn /deln
-'
-
-bashbot_examples="${bashbot_title}"'
-
-'
-
-bashbot_group='*#MD Amazon Deutschland only Deals*
-https://t.me/joinchat/IvvRtlEPhZKmWVVc8iTD2g
-
-*#MD Zuhause, Family & Kids und Empfehlungen*
-https://t.me/joinchat/IvvRtlT4f8HuwFDiJDCWoQ '
-
-bashbot_all="${bashbot_title}"'
-
----------------------------------------------------------------------
-'"${bashbot_group}"'
-
-*#DL Amazon International*
-https://t.me/joinchat/IvvRtkzPaMTHFZi3CjSX8g
-
-*#MD Amazon Warehouse Schnäppchen *
-https://t.me/joinchat/IvvRthRhj5NVkcihTxDZwQ
-
-----------------------------------------------------------------------
-Was hier kommt, kommt nur hier!
-
-*#MD Kleidung, Accessoirs & Empfehlungen*
-https://t.me/joinchat/IvvRthlJ8zwC7tHyhl63Fg
-
-*#MD Mobilfunk & Smartphones*
-https://t.me/joinchat/IvvRthtqth9Svyyvnb5lAQ
-
-*#MD Gaming, Spiele & Konsolen*
-https://t.me/joinchat/IvvRthRyrsmIyMFMc7189Q
-
----------------------------------------------------------------------
-*#MD extrem - All Dealz über 100°*
-https://t.me/joinchat/IvvRtk6ehHS4ZZmtoRFU2g
-
-*#MD Offtopic: Diskussion & Ankündigungen*
-https://t.me/joinchat/IvvRthRhMcX6rDQU-pZrWw
-
-*Du willst Dealz suchen oder abbonieren?*
-Klicke hier -> @'"${ME//_/\\\\_}"' <- und schicke /start
-.
+Du hast eine eigene Telegram Dealgruppe? Lade mich ein und alle in der Guppe können mich nutzen:
 '
 
 # uncomment the following lines to overwrite info and help messages
 bashbot_info="${bashbot_title}"'
 
-'"${bashbot_group}"'
-
-*#MD Offtopic: Diskussion & Ankündigungen*
-Hier kannst Du dich austauschen und erfährst Neues.
-https://t.me/joinchat/IvvRthRhMcX6rDQU-pZrWw
-
 *Du willst Dealz suchen oder tracken?*
-Klicke hier -> @'"${ME//_/\\\\_}"' <- und schicke /start
+Klicke hier https://t.me/'"${ME//_/\\_}"' und schicke /start
 
-*Meine Befehle*:
-  /suche     _- Suchen und Tracken von Dealz_
-  /watch     _- Beobachten von Amazon Artikeln_
-  /gruppen  _- Liste aller Dealz Gruppen_
-  /hilfe    _- Hilfe zu weiteren Befehlen_
+'"${bashbot_commands}"'
 
 *Dein '"${ME//_/-}"'*
-https://dealz.rrr.de/amzdealz.html
 '
 
 bashbot_help="${bashbot_title}"'
 
-Du hast eine eigene Telegram Dealgruppe? Lade mich ein und all in der Guppe können mich nutzen:
+*Du willst mich weiterempfehlen?*
+Schicke https://t.me/'"${ME//_/\\_}"' an deinen Telegram Kontakt und sage ihm er soll darauf klicken und /start schicken.
 
-*verfügbare Befehle*:
- /suche     _- Suchen und Tracken von Dealz_
- /watch     _- Beobachten von Amazon Artikeln_
- /gruppen  _- Liste aller Dealz Gruppen_
+'"${bashbot_commands}"'
 
- /info _- Info zum Bot_
- /help _- Zeigt diese Liste_
-
-_https://dealz.rrr.de/assets/images/rdealomat.gif_
+*Dein '"${ME//_/-}"'*
 '
 
 # in a regular message is no need to escape '_'
 SORRYADMIN="Du bist kein Gruppenadmin, diesen Befehl kannst Du nur im privaten Chat ausführen @${ME} <<- hier klicken"
-SORRYADMIN2="Dieser Befehl ist dem Botadmin vorbehalten, sorry."
 
 # Set INLINE to 1 in order to receive inline queries.
 # To enable this option in your bot, send the /setinline command to @BotFather.
-INLINE=""
+INLINE="0"
+
+# if your bot is group admin it get commands sent to other bots
+# Set MEONLY to 1 to ignore commands sent to other bots
+export MEONLY="1"
+
 # Set to .* to allow sending files from all locations
-FILE_REGEX='/home/deal-O-mat/.*'
+FILE_REGEX='/this_is_my_bot_path/.*'
 # run curl over TOR or SOCKS
 #export BASHBOT_CURL_ARGS="--socks5-hostname localhost"
 
@@ -180,40 +103,32 @@ WELCOME_MSG="Willkommen"
 export REPORT_NEWMEMBER="yes"
 export REPORT_LEFTMEMBER="yes"
 
-export REMINDER="Del-O-Mat: https://dealz.rrr.de/amzdealz.html\n https://dealz.rrr.de/ebaydealz.html"
-
+# location of Database files
 TRACKFILE="${DATADIR}/0-dealtrack"
 SEARCHFILE="${DATADIR}/0-dealsearch"
 WATCHFILE="${DATADIR}/0-dealwatch"
-NOTIFYFILE="${DATADIR}/0-dealnotify"
 
 if [ "$1" = "startbot" ];then
-    # run once after startup when the first message is received
-    my_startup(){
-	# send reminder on startup, random delay
-	send_normal_message "-10011894xxxxx" "${REMINDER}"
-    }
+    # mark startup, triggers action on first message
     setConfigKey "startupaction" "await"
-    # create KEYOFILE DB if not exist
+    # create Database files on startup
     jssh_newDB "${TRACKFILE}"
     jssh_newDB "${SEARCHFILE}"
     jssh_newDB "${WATCHFILE}"
-    jssh_newDB "${NOTIFYFILE}"
 else
-    # things to do only at source, eg. after startup
-   if [[ "$(getConfigKey "startupaction")" != "done"* ]]; then
-	setConfigKey "startupaction" "done $(date)"
-	my_startup
-    fi
 
-    # your additional bashbot commands
+    # your additional bahsbot commands
     # NOTE: command can have @botname attached, you must add * in case tests... 
     mycommands() {
+	# action triggered on first message after startup
+	if [[ "$(getConfigKey "startupaction")" != "done"* ]]; then
+		setConfigKey "startupaction" "done $(date)"
+   	fi
 	# a service Message was received
 	local SILENCER="yes"
 	if [[ "${SERVICE}" != "" ]]; then
 		# example: dleted service messages
-		if [ "${SILENCER}" = "yes" ]; then
+		if [ "${SILENCER}" = "yes" ] && bot_is_admin "${CHAT[ID]}"; then
 			delete_message "${CHAT[ID]}" "${MESSAGE[ID]}"
 		fi
 	fi
@@ -222,46 +137,35 @@ else
 	[ -n "${REMOVEKEYBOARD}" ] && remove_keyboard "${CHAT[ID]}" "..." &
 	[[ -n "${REMOVEKEYBOARD_PRIVATE}" &&  "${CHAT[ID]}" == "${USER[ID]}" ]] && remove_keyboard "${CHAT[ID]}" "..." &
 
-	######################
-	# default commands
 	# fix upper case first letter in commands
 	[[ "${MESSAGE}" =~  ^/[[:upper:]] ]] && MESSAGE="${MESSAGE:0:1}$(tr '[:upper:]' '[:lower:]' <<<"${MESSAGE:1:1}")${MESSAGE:2}"
 
+	######################
+	# default commands
 	case "${MESSAGE}" in
 		'/info'*)
 			send_action "${CHAT[ID]}" "typing"
 			delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
-			return 0 # continue with default action
+			send_markdownv2_message "${CHAT[ID]}" "${bashbot_info}"
+			return 1 # continue with default action
 			;;
 		'/hel'*|'/hil'*)
 			send_action "${CHAT[ID]}" "typing"
 			delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
-			send_markdown_message "${CHAT[ID]}" "${bashbot_help}"
+			send_markdownv2_message "${CHAT[ID]}" "${bashbot_help}"
 			return 1 # break, do not continue
 			;;
 		'/start'*)
-			send_markdown_message "${CHAT[ID]}" "${bashbot_info}"
+			send_markdownv2_message "${CHAT[ID]}" "${bashbot_help}"
 			return 1 # break, do not continue
 			;;
-		'/gr'*) # list groups
-			delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
-			send_action "${CHAT[ID]}" "typing"
-			send_markdown_message "${CHAT[ID]}" "${bashbot_all}"
-			return
-			;;
-		#'/test'*)
-		#	send_normal_message "${CHAT[ID]}" "Start interactive"
-		#	send_markdown_message "${CHAT[ID]}" "TEST: äöüß!^°_-\"-,§$%&/(){}#@?[]{}._"
-		#	return
-		#	;;
-		# will we process edited messages also?
 		'/_edited_message'*)
 			#return 1 # no
 			# but if we do, remove /edited_message
 			MESSAGE="${MESSAGE#/* }"
 			;;
 		'/_new_chat_member'*)
-			if [[ -n "${WELCOME_NEWMEMBER}" && "${NEWMEMBER[ISBOT]}" != "true" ]]; then
+			if [[ -n "${WELCOME_NEWMEMBER}" && "${NEWMEMBER[ISBOT]}" != "true" ]] && bot_is_admin "${CHAT[ID]}"; then
 			    send_normal_message "${CHAT[ID]}"\
 				"${WELCOME_MSG} ${NEWMEMBER[FIRST_NAME]} ${NEWMEMBER[LAST_NAME]} (@${NEWMEMBER[USERNAME]})"
 			    MYSENTID="${BOTSENT[ID]}"
@@ -279,8 +183,8 @@ else
 	##########
 	# pre test for admin only commands
 	case "${MESSAGE}" in
-		# must be in private chat, group admin, or botadmin
-		'/sea'*|'/su'*|'/add'*|'/new'*|'/del'*|'/tra'*|'/wat'*|'/noti'*|'/lista'*)
+		# must be private, group admin, or botadmin
+		'/sea'*|'/su'*|'/add'*|'/new'*|'/del'*|'/tra'*)
 			send_action "${CHAT[ID]}" "typing"
 			if ! user_is_admin "${CHAT[ID]}" "${USER[ID]}" ; then
 			    delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
@@ -289,21 +193,13 @@ else
 			    { sleep 5; delete_message  "${CHAT[ID]}" "${MYSENTID}"; } &
 			    return 1
 			fi
-			;;&
-		# must be botadmin
-		'/delu'*) 
-			if ! user_is_botadmin "${USER[ID]}" ; then
-			    delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
-			    send_markdown_message "${CHAT[ID]}" "*${SORRYADMIN2}*";
-			    MYSENTID="${BOTSENT[ID]}"
-			    { sleep 5; delete_message  "${CHAT[ID]}" "${MYSENTID}"; } &
-			    return 1
-			fi
+			# ok, now lets process the real command 
 			;;
 	esac
 
 	#################
 	# search commands
+	#local vairable, e.g. read/write Database
 	local FINDKEYS OLDKEYS KEY MYSENTID
 	declare -a KEYARR
  	declare -A SEARCHVALUES
@@ -311,69 +207,30 @@ else
 	#################
 	# pre processing of search commands
 	case "${MESSAGE}" in
-		'/add'*|'/tra'*|'/noti'*) # add no arg
+		'/add'*|'/tra'*) # add no arg
 			FINDKEYS="${MESSAGE#/* }"
 			if [ "${FINDKEYS}" = "${MESSAGE}" ]; then 
 			    send_normal_message "${CHAT[ID]}" "Kein Begriff angegeben!"
 			    exit
 			fi
 			;;&
-		'/addw'*|'/wat'*) OLDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${WATCHFILE}")"
-			;;&
 		'/add'[kbt]*|'/tra'*) OLDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${TRACKFILE}")"
 			;;&
-		'/noti'*) OLDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${NOTIFYFILE}")"
-			;;&
-
 		'/addk'*|'/addt'*|'/tra'*) # add track
 			MESSAGE="/newtrack ${FINDKEYS} ${OLDKEYS}"
 			;;
-		'/addb'*) # add black
-			[[ "${OLDKEYS}" != *'!'* ]] && FINDKEYS="!${FINDKEYS}"
-			MESSAGE="/newtrack ${OLDKEYS} ${FINDKEYS}"
-			;;
-		'/addw'*|'/wat'*) # add watch
-			FINDKEYS="${MESSAGE#/* }"
-			if [ "${FINDKEYS}" = "${MESSAGE}" ]; then 
-			    delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
-			    send_markdown_message "${CHAT[ID]}" "${bashbot_whelp}"
-			    return
-			fi
-			MESSAGE="/newwatch ${FINDKEYS} ${OLDKEYS}"
-			;;
-		'/noti'*) # add watch
-			MESSAGE="/newnotify ${FINDKEYS} ${OLDKEYS}"
-			;;
-		'/delk'*|'/delb'*) # no user search
+		'/delk'*) #delete key
 			OLDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${TRACKFILE}")"
 			if [ "${OLDKEYS}" = "" ]; then
-			    send_markdown_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"
+			    send_markdownv2_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"
 			    return
 			fi
-			;;&
-		'/delw'*) # no watch
-			OLDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${WATCHFILE}")"
-			if [ "${OLDKEYS}" = "" ]; then
-			    send_markdown_message "${CHAT[ID]}" "*Kein Product Watch aktiv!*"
-			    return
-			fi
-			;;&
-		'/deln'*) # no notify
-			OLDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${NOTIFYFILE}")"
-			if [ "${OLDKEYS}" = "" ]; then
-			    send_markdown_message "${CHAT[ID]}" "*Kein Product Notify aktiv!*"
-			    return
-			fi
-			jssh_deleteKeyDB "${CHAT[ID]}" "${NOTIFYFILE}"
-			send_normal_message "${CHAT[ID]}" "Product Notify \"${OLDKEYS}\" gelöscht!"
-			;;
-		'/delk'*) #delete key
 			KEY="${OLDKEYS%%|!*}"
 			FINDKEYS="${MESSAGE#/* }"
 			read -r -a KEYARR <<<"ALL ${KEY//|/ }"
 			if [ "${FINDKEYS}" = "ALL" ]; then
 				jssh_deleteKeyDB "${CHAT[ID]}" "${TRACKFILE}"
-				send_markdown_message "${CHAT[ID]}" "*Tracking gelöscht!*"
+				send_markdownv2_message "${CHAT[ID]}" "*Tracking gelöscht!*"
 				return
 			elif [[ "${FINDKEYS}" =~ ^[0-9]+$ ]]; then
 			    if [ "${#KEYARR[@]}" -lt "${FINDKEYS}" ]; then
@@ -386,7 +243,7 @@ else
 				FINDKEYS="${KEYARR[*]}"
 				if [ -z "${FINDKEYS}" ]; then
 					jssh_deleteKeyDB "${CHAT[ID]}" "${TRACKFILE}"
-					send_markdown_message "${CHAT[ID]}" "*Tracking gelöscht!*"
+					send_markdownv2_message "${CHAT[ID]}" "*Tracking gelöscht!*"
 					return
 				else
 					KEY="${OLDKEYS#*!}"
@@ -395,80 +252,15 @@ else
 				fi
 			    fi
 			else
-			    OUT="Keys:\n$(printKeys "KEYARR")\n\nSchicke \"/delkey <Nr.>\" zum Löschen."
+			    OUT="$(printKeys "KEYARR")\n\nSchicke \"/delkey <Nr.>\" zum Löschen."
+			    # send keyboard in private chats only
 			    if [ "${CHAT[ID]}" != "${USER[ID]}" ]; then
 				send_normal_message "${CHAT[ID]}" "${OUT}"
 			    else
 				send_keyboard "${CHAT[ID]}" "${OUT}"\
 				'["/delkey 1","/delkey 2","/delkey 3"],["/delkey 4","/delkey 5","/delkey 6"],["/delkey 7","/delkey 8","/delkey 9"],["/delkey ALL","/delblack","/listtrack"]'
 			    fi
-			fi
-			;;
-		'/delb'*) #delete black
-			KEY="${OLDKEYS#*!}"
-			read -r -a KEYARR <<<"ALL ${KEY//|/ }"
-			FINDKEYS="${MESSAGE#/* }"
-			if [ "${KEY}" = "${OLDKEYS}" ]; then
-				unset FINDKEYS
-				unset KEYARR
-				KEYARR[0]="Keine Blacklist"
-			fi
-			if [[ "${FINDKEYS}" =~ ^[0-9]+$ ]]; then
-			    if [ "${#KEYARR[@]}" -lt "${FINDKEYS}" ]; then
-				send_normal_message "${CHAT[ID]}" "Es gibt nur $((${#KEYARR[@]}-1)) Keys, bitte nochmal."
-				unset "KEYARR[${FINDKEYS}]"
-			    else
-				send_normal_message "${CHAT[ID]}" "Lösche Black ${KEYARR[${FINDKEYS}]}"
-				unset "KEYARR[0]"
-				unset "KEYARR[${FINDKEYS}]"
-				FINDKEYS="|!${KEYARR[*]}"
-				[ "${FINDKEYS}" == "|!" ] && FINDKEYS=""
-				MESSAGE="/newt ${OLDKEYS%%|!*}${FINDKEYS}"
-			    fi
-			fi
-			if [[ -z "${FINDKEYS}" || "${FINDKEYS:0:1}" == "/" ]]; then # output list
-			    OUT="Blacklist:\n$(printKeys "KEYARR")\n\nSchicke \"/delblack <Nr.>\" zum Löschen."
-			    if [ "${CHAT[ID]}" != "${USER[ID]}" ]; then
-				send_normal_message "${CHAT[ID]}" "${OUT}"
-			    else
-				send_keyboard "${CHAT[ID]}" "${OUT}"\
-				'["/delblack 1","/delblack 2","/delblack 3"],["/delblack 4","/delblack 5","/delblack 6"],["/delblack 7","/delblack 8","/delblack 9"],["/delblack ALL","/delkey","/listtrack"]'
-			    fi
-			fi
-			;;
-		'/delw'*) #delete watch
-			KEY="${OLDKEYS}"
-			FINDKEYS="${MESSAGE#/* }"
-			read -r -a KEYARR <<<"ALL ${KEY//|/ }"
-			if [ "${FINDKEYS}" = "ALL" ]; then
-				jssh_deleteKeyDB "${CHAT[ID]}" "${WATCHFILE}"
-				send_markdown_message "${CHAT[ID]}" "*Produkt Watch gelöscht!*"
-				return
-			elif [[ "${FINDKEYS}" =~ ^[0-9]+$ ]]; then
-			    if [ "${#KEYARR[@]}" -lt "${FINDKEYS}" ]; then
-				send_normal_message "${CHAT[ID]}" "Es gibt nur $((${#KEYARR[@]}-1)) Keys, bitte nochmal."
-				unset FINDKEYS
-			    else
-				send_normal_message "${CHAT[ID]}" "Lösche Key ${KEYARR[${FINDKEYS}]}"
-				unset "KEYARR[0]"
-				unset "KEYARR[${FINDKEYS}]"
-				FINDKEYS="${KEYARR[*]}"
-				if [ -z "${FINDKEYS}" ]; then
-					jssh_deleteKeyDB "${CHAT[ID]}" "${WATCHFILE}"
-					send_markdown_message "${CHAT[ID]}" "*Produkt Watch gelöscht!*"
-					return
-				else
-					MESSAGE="/neww ${FINDKEYS}"
-				fi
-			    fi
-			else
-			    OUT="Keys:\n$(printKeys "KEYARR")\n\nSchicke \"/delwatch <Nr.>\" zum Löschen."
-			    if [ "${CHAT[ID]}" != "${USER[ID]}" ]; then
-				send_normal_message "${CHAT[ID]}" "${OUT}"
-			    else
-				send_keyboard "${CHAT[ID]}" "${OUT}"\
-				'["/delwatch 1","/delwatch 2","/delwatch 3"],["/delwatch 4","/delwatch 5","/delwatch 6"],["/delwatch 7","/delwatch 8","/delwatch 9"],["/delwatch ALL","/listwatch"]'
-			    fi
+			    return 1
 			fi
 			;;
 	esac
@@ -480,9 +272,9 @@ else
 			FINDKEYS="${MESSAGE#/* }"
 			if [ "${FINDKEYS}" = "${MESSAGE}" ]; then 
 			    delete_message  "${CHAT[ID]}"  "${MESSAGE[ID]}"
-			    send_markdown_message "${CHAT[ID]}" "${bashbot_shelp}"
+			    send_markdownv2_message "${CHAT[ID]}" "${bashbot_shelp}"
 			else
-			  if user_is_admin "${CHAT[ID]}" "${USER[ID]}" ; then
+			  if user_is_admin "${CHAT[ID]}" "${USER[ID]}" || user_is_allowed "${USER[ID]}" "search" "${CHAT[ID]}"; then
 			    set_keys "${FINDKEYS}" "${SEARCHFILE}"
 			  else
 			    send_normal_message "${CHAT[ID]}" "${SORRYADMIN}"
@@ -491,15 +283,15 @@ else
 			    return 1
 			  fi
 			fi
-			return
+			return 1
 			;;
 
 		'/newt'*) # newtrack
 			FINDKEYS="${MESSAGE#/* }"
 			FINDKEYS="${FINDKEYS%[\" ]}"
 			FINDKEYS="${FINDKEYS#[\" ]}"
-			if [ "${FINDKEYS}" = "${MESSAGE}" ]; then 
-			    send_markdown_message "${CHAT[ID]}" "${bashbot_shelp}"
+			if [ "${FINDKEYS}" = "" ]; then 
+			    send_markdownv2_message "${CHAT[ID]}" "${bashbot_shelp}"
 			else
 			    set_keys "${FINDKEYS}" "${TRACKFILE}"
 			fi
@@ -508,181 +300,66 @@ else
 			send_action "${CHAT[ID]}" "typing"
 			FINDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${TRACKFILE}")"
 			if [ "${FINDKEYS}" = "" ]; then
-			    send_markdown_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"
+			    send_markdownv2_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"
 			    return
 			fi
 			OUT="Tracking nach \"${FINDKEYS}\" ist aktiv."
+			# send keyboard in private chats only
 			if [[ "${CHAT[ID]}" != "${USER[ID]}" ]]; then
 			    send_normal_message "${CHAT[ID]}" "${OUT}"
 			else
 			    send_keyboard "${CHAT[ID]}" "${OUT}"\
-			     '["/delkey 1","/delkey 2","/delkey 3"],["/delkey 4","/delkey 5","/delblack 6"],["/delblack 1","/delblack 2","/delblack 3"],["/delkey","/delblack","/listtrack"]'
+			     '["/delkey 1","/delkey 2","/delkey 3"],["/delkey 4","/delkey 5","/delkey 6"],["/delkey 7","/delkey 8","/delkey 9"],["/delkey ALL","/delblack","/listtrack"]'
 			fi
-			return
+			return 1
 			;;
-		'/neww'*) # newwatch
-			FINDKEYS="${MESSAGE#/* }"
-			FINDKEYS="${FINDKEYS%[\" ]}"
-			FINDKEYS="${FINDKEYS#[\" ]}"
-			if [ "${FINDKEYS}" = "${MESSAGE}" ]; then 
-			    send_markdown_message "${CHAT[ID]}" "${bashbot_shelp}"
-			else
-			    set_keys "${FINDKEYS}" "${WATCHFILE}"
-			fi
-			;&
-		'/listw'*) # listwatch
-			send_action "${CHAT[ID]}" "typing"
-			FINDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${WATCHFILE}")"
-			if [ "${FINDKEYS}" = "" ]; then
-			    send_markdown_message "${CHAT[ID]}" "*Kein Produkt Watch aktiv!*"
-			    return
-			fi
-			OUT="Produkt Watch nach \"${FINDKEYS}\" ist aktiv."
-			if [[ "${CHAT[ID]}" != "${USER[ID]}" ]]; then
-			    send_normal_message "${CHAT[ID]}" "${OUT}"
-			else
-			    send_keyboard "${CHAT[ID]}" "${OUT}"\
-			     '["/delwatch 1","/delwatch 2","/delwatch 3"],["/delwatch 4","/delwatch 5","/delwatch 6"],["/delwatch 7","/delwatch 8","/delwatch 9"],["/delwatch","/delwatch ALL","/listwatch"]'
-			fi
-			return
-			;;
-		'/newn'*) # newnotify
-			FINDKEYS="${MESSAGE#/* }"
-			FINDKEYS="${FINDKEYS%[\" ]}"
-			FINDKEYS="${FINDKEYS#[\" ]}"
-			if [ "${FINDKEYS}" = "${MESSAGE}" ]; then 
-			    send_markdown_message "${CHAT[ID]}" "${bashbot_shelp}"
-			else
-			    set_keys "${FINDKEYS}" "${NOTIFYFILE}"
-			fi
-			;&
-		'/listn'*) # listnotify
-			send_action "${CHAT[ID]}" "typing"
-			FINDKEYS="$(jssh_getKeyDB "${CHAT[ID]}" "${NOTIFYFILE}")"
-			if [ "${FINDKEYS}" = "" ]; then
-			    send_markdown_message "${CHAT[ID]}" "*Kein Produkt Notify aktiv!*"
-			    return
-			fi
-			OUT="Produkt Notify nach \"${FINDKEYS}\" ist aktiv."
-			send_normal_message "${CHAT[ID]}" "${OUT}"
-			return
-			;;
-
-		#############################
-		# botadmin only commands
-		'/listat'*) # listalltrack
-			jssh_readDB "SEARCHVALUES" "${TRACKFILE}"
-			FINDKEYS="$(jssh_printDB "SEARCHVALUES")"
-			if [ "${FINDKEYS}" != "" ]; then
-			    # shellcheck disable=SC2126
-			    send_action "${CHAT[ID]}" "typing"
-			    send_normal_message "${CHAT[ID]}" "All Suchaufträge:\n${FINDKEYS}"
-			else
-			    send_markdown_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"
-			fi
-			;;
-		'/listaw'*) # listall watch
-			jssh_readDB "SEARCHVALUES" "${WATCHFILE}"
-			FINDKEYS="$(jssh_printDB "SEARCHVALUES")"
-			if [ "${FINDKEYS}" != "" ]; then
-			    send_action "${CHAT[ID]}" "typing"
-			    send_normal_message "${CHAT[ID]}" "Alle Watchaufträge:\n${FINDKEYS}"
-			else
-			    send_markdown_message "${CHAT[ID]}" "*Kein Produkt Watch aktiv!*"
-			fi
-			return
-			;;
-		'/delu'*) # delusersearch
-			KEY="${MESSAGE#/* }"; KEY="${KEY#%% *}"
-			if [ "${KEY}" = "${MESSAGE}" ] || [ "${KEY}" = "" ] ; then
-			    jssh_readDB "SEARCHVALUES" "${TRACKFILE}"
-			    FINDKEYS="$(jssh_printDB "SEARCHVALUES")"
-			    if [ "${FINDKEYS}" != "" ]; then
-				send_normal_message "${CHAT[ID]}"\
-					"Aktive Suchaufträge:\n${FINDKEYS}\n\nSende \"/deluser ID\" zum Löschen eines Users"
-			    else
-				send_markdown_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"; exit
-			    fi
-			    exit
-			fi
-			jssh_deleteKeyDB "${KEY}" "${TRACKFILE}"
-			send_normal_message "${CHAT[ID]}" "Lösche Suchauftrag User ${KEY}"
-			jssh_readDB "SEARCHVALUES" "${TRACKFILE}"
-			FINDKEYS="$(jssh_printDB "SEARCHVALUES")"
-			if [ "${FINDKEYS}" != "" ]; then
-			    send_normal_message "${CHAT[ID]}" "Verbliebene Suchaufträge:\n${FINDKEYS}"
-			else
-			    send_markdown_message "${CHAT[ID]}" "*Kein Tracking aktiv!*"; exit
-			fi
-			return
-			;;
-
 	esac
-
      }
 
 fi
 
-    # debug function called on start, stop of bot, interactive and  background processes
-    # if your bot was started with debug as second argument
-    # $1 current date, $2 from where the function wqs called, $3 ... $n optional information
-    my_debug_checks() {
-	# example check because my bot creates a wrong file, this was becuase an empty variable
-	[ -f ".jssh" ] && printf "%s: %s\n" "${1}" "Ups, found file \"${PWD:-.}/.jssh\"! =========="
-    }
-
-    # called when bashbot sedn command failed because we can not connect to telegram
-    # return 0 to retry, return non 0 to give up
-    bashbotBlockRecover() {
-	# place your commnds to unblock here, e.g. change IP or simply wait
-	sleep 60 && return 0 # may be temporary
-	return 1 
-    }
-
     # place your processing functions here
     set_keys(){
-	local MYFIND MYKEY MYKEYF MINLEN MYCHECK MYSEARCH
+	local MYFIND MYKEY MYKEYF MINLEN MYSEARCH
         declare -A KEYARR
 	MYFIND="$1"
 	MYKEYF="$2"
-	MINLEN="3"
+	MINLEN="4"
 	# check len of keys
-	for MYKEY in ${MYFIND}; do [ "${#MYKEY}" -lt ${MINLEN} ] && break; done
-	if [ "${#MYKEY}" -lt ${MINLEN} ]; then
-		send_markdown_message "${CHAT[ID]}" "*Ein Suchbegriff ist kürzer als ${MINLEN} Zeichen!*"
+	for MYKEY in ${MYFIND}; do [ "${#MYKEY}" -lt "${MINLEN}" ] && break; done
+	if [ "${#MYKEY}" -lt "${MINLEN}" ]; then
+		send_markdownv2_message "${CHAT[ID]}" "*Ein Suchbegriff ist kürzer als ${MINLEN} Zeichen!*"
 	else
 		MYFIND="$(create_pattern "${MYFIND}")"
-		MYCHECK="$(check_pattern "${MYFIND}")"
 		[[ "${MESSAGE}" == "/s"* ]] &&\
-			send_normal_message "${CHAT[ID]}" "${MYCHECK}Suche nach \"${MYFIND//|/ }\" wird gestartet ..."
+			send_normal_message "${CHAT[ID]}" "Suche nach \"${MYFIND//|/ }\" wird gestartet ..."
 		MYKEY="${MYFIND//[^|]}"
 		if [ "${#MYKEY}" -gt 49 ]; then
-		    send_markdown_message "${CHAT[ID]}" "*Maximale Anzahl von 50 Begriffen erreicht!*"
+		    send_markdownv2_message "${CHAT[ID]}" "*Maximale Anzahl von 50 Begriffen erreicht!*"
 		elif [ "${MYFIND}" != "" ]; then
 		    KEYARR["${CHAT[ID]}"]="${MYFIND}"
 		    jssh_updateDB "KEYARR" "${MYKEYF}"
-		    #jssh_insertDB "${CHAT[ID]}" "${MYFIND}" "${MYKEYF}"
 		else
-		    send_markdown_message "${CHAT[ID]}" "*Ein Begriff ist ungültig, z.B. \" , oder leer!*"
+		    send_markdownv2_message "${CHAT[ID]}" "*Ein Begriff ist ungültig, z.B. \" , oder leer!*"
 		fi
 	fi
     }
     # place your processing functions here
 
-
-    # $1 ARRAYNAME $2 command
-    printKeys() {
+# $1 ARRAYNAME to print
+printKeys() {
 	local key
 	declare -n ARRAY="$1"
 	for key in "${!ARRAY[@]}"
        	do
 		printf '%s  -  %s\n' "${key}" "${ARRAY[${key}]}"
        	done
-    }
+}
 
-    # create a regex from space sepeareted keywords
-    # $1 space separated words
-    create_pattern() {
+
+# create a regex from space sepeareted keywords
+# $1 space separated words
+create_pattern() {
 	local PATTERN KEY
 	set -f
 	for KEY in $1
@@ -690,28 +367,7 @@ fi
 		[ "${PATTERN}" != "" ] && PATTERN="${PATTERN}|"
 		PATTERN="${PATTERN}${KEY}"
 	done
-	# revome blank and "
-	PATTERN="${PATTERN//[\" ]}"
-	# remove * at start
-	PATTERN="${PATTERN//|\*/|}"; PATTERN="${PATTERN#\*}"
-	# remove unneeded |
-	PATTERN="${PATTERN//||/|}"; PATTERN="${PATTERN#|}"; PATTERN="${PATTERN%|}"
 	set +f
 	echo "${PATTERN}"
-    }
+}
 
-    # check regex for common errors
-    # ¤1 pattern
-    check_pattern() {
-	local WARN=""
-	if [[ "$1" =~ ([^.\)]\*)|(^\*) ]]; then
-		WARN+="Meintest du evtl. '.*' ?  '*' ist vorheriges Zeichen beliebig oft. "$'\n\r'
-	fi
-	if [[ "$1" =~ (^\*)|(^.\*)|([^\)]\*\|)|(\|\*)|(\|\.\*)|([^\)]\*$) ]] ; then
-		WARN+="Ein '.*' oder '*' an Anfang oder End ist unnötig. "$'\n\r'
-	fi
-	if [[ "$1" =~ \([^|\)]*\| ]] ; then
-		WARN+="Öffnende '(' ohne ')' gefunden. Klammerausdruck muss vor '|' geschlossen sein. "$'\n\r'
-	fi
-	[ -n "${WARN}" ] && printf "Potentielle Fehlerquellen:\n\r%s\n\r" "${WARN}"
-    }

examples/notify.sh

@@ -1,17 +1,26 @@
 #!/bin/bash
-# file: notify.sh
-# example for an background job, run with startback notify.sh
+########################################################################
+#
+# File: notify.sh
+#
+# Description: example for an background job, see mycommands.sh.dist
+#
+# Usage: runback notify  example/notify.sh [seconds] - or run in terminal
+#        killback notify - to stop background job
+#
+# Options: seconds - time to sleep between output, default 10
 #
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
-#### $$VERSION$$ v1.0-0-g99217c4
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+########################################################################
 
 ######
 # parameters
 # $1 $2 args as given to starct_proc chat script arg1 arg2
 # $3 path to named pipe/log
 
-
 # adjust your language setting here
 # https://github.com/topkecleon/telegram-bot-bash#setting-up-your-environment
 export 'LC_ALL=C.UTF-8'
@@ -24,18 +33,20 @@ unset IFS
 # discard STDIN for background jobs!
 cat >/dev/null & 
 
-# check if $1 is a number
-re='^[0-9]+$'
-if [[ $1 =~ $re ]] ; then
+# $1 = time between time notifications
+# check if $1 is a valid number
+if [[ "$1" =~ ^[0-9]+$ ]] ; then
 	SLEEP="$1"
 else
-	SLEEP=10 # time between time notifications
+	SLEEP=10
 fi
 
 # output current time every $1 seconds
-date "+* It's %k:%M:%S o' clock ..."
-while sleep $SLEEP
+printf "Output time every %s seconds ...\n" "${SLEEP}"
+
+while true 
 do
-	date "+* It's %k:%M:%S o' clock ..."
+	date "+* It's %k:%M:%S o'clock ..."
+	sleep "${SLEEP}"
 done
 

examples/question.sh

@@ -1,11 +1,17 @@
 #!/bin/bash
-# file: question.sh
-# example for an interactive chat, run with startproc question.sh
+########################################################################
+#
+# File: question.sh
+#
+# Usage: runproc example/question.sh - or run in terminal
+#
+# Description: example for an interactive chat, see mycommands.sh.dist
 #
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
+########################################################################
 
 ######
 # parameters
@@ -14,7 +20,6 @@
 
 INPUT="${3:-/dev/stdin}"
 
-
 # adjust your language setting here
 # https://github.com/topkecleon/telegram-bot-bash#setting-up-your-environment
 export 'LC_ALL=C.UTF-8'
@@ -24,18 +29,29 @@ export 'LANGUAGE=C.UTF-8'
 unset IFS
 # set -f # if you are paranoid use set -f to disable globbing
 
-echo "Why hello there.
-Would you like some tea (y/n)?"
+# kill interactive script if not finished in time, e.g. user away or error
+MAXWAIT="1m"
+{ sleep "${MAXWAIT}"; printf "Stopping Questionnaire after %s, you need to much time to finish ... BYE\n" "${MAXWAIT}"; kill $$; wait 2>/dev/null ;} &
+
+# simple yes/no question, defaults to no
+printf "Hi, hello there\nWould you like some tea (y/n)?\n"
 read -r answer <"${INPUT}"
-[[ $answer =~ ^([yY][eE][sS]|[yY])$ ]] && echo "OK then, here you go: http://www.rivertea.com/blog/wp-content/uploads/2013/12/Green-Tea.jpg" || echo "OK then."
-until [ "$SUCCESS" = "y" ] ;do
-	echo 'Do you like Music? mykeyboardstartshere "Yass!" , "No"'
+if [[ ${answer,,} == "y"* ]]; then
+	printf "OK then, here you go: http://www.rivertea.com/blog/wp-content/uploads/2013/12/Green-Tea.jpg\n"
+else
+	printf "OK then, no tea ...\n"
+fi
+
+# question with Keyboard, repeating until correct answer given
+until [ "${SUCCESS}" = "y" ] ;do
+	printf 'Do you like Music? mykeyboardstartshere "Yass!" , "No"\n'
 	read -r answer <"${INPUT}"
-	case $answer in
-		'Yass!') echo "Goody! mykeyboardendshere";SUCCESS=y;;
-		'No') echo "Well that's weird. mykeyboardendshere";SUCCESS=y;;
-		'') echo "empty answer!" && exit ;;
+	case ${answer,,} in
+		'') printf "empty answer! Try again\n";; 
+		'yass'*) printf "Goody! mykeyboardendshere\n";SUCCESS=y;;
+		'no'*) printf "Well that's weird. mykeyboardendshere\n";SUCCESS=y;;
 		*) SUCCESS=n;;
 	esac
 done
+printf "OK, Done!\n"
 

examples/send-system-status/botacl

@@ -1,7 +1,7 @@
 # file: botacl
 # a user not listed here, will return false from 'user_is_allowed'
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 # Format:
 # user:resource:chat
 

examples/send-system-status/mycommands.sh

@@ -5,7 +5,7 @@
 # to show how you can customize bashbot by only editing mycommands.sh
 # NOTE: this is not tested, simply copied from original source and reworked!
 #
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 #
 # shellcheck disable=SC2154
 # shellcheck disable=SC2034
@@ -38,7 +38,7 @@ mycommands() {
     local msg=""
 
     if user_is_botadmin "${USER[ID]}" || user_is_allowed "${USER[ID]}" "systemstatus"; then
-	case "$CMD" in
+	case "${CMD}" in
 		'/md'*) msg="$(cat /proc/mdstat)";;
 		'/smb'*) msg="$(smbstatus)" ;;
 		'/se'*) msg="$(sensors | sed -r 's/\s|\)+//g' | sed -r 's/\(high=|\(min=/\//' | sed -r 's/\,crit=|\,max=/\//')";;
@@ -51,14 +51,14 @@ mycommands() {
 		'/smart'*)
 			[ "${CMD[1]}" == "" ] && msg="example \`/smart sda\`" && return
 			drive="$(echo "${CMD[1]}" | cut -c 1-3)"
-			echo "smartctl -a /dev/$drive"
-			msg="$(smartctl -a "/dev/$drive")"
+			echo "smartctl -a /dev/${drive}"
+			msg="$(smartctl -a "/dev/${drive}")"
 			;;
 		'/df') msg="$(df -h | sed -r 's/^/\n/' | sed -r 's/\s+/\n/g')";;
 	esac
 
-	if [ "$msg" != "" ]; then
-		send_normal_message "${CHAT[ID]}" "$msg" 
+	if [ "${msg}" != "" ]; then
+		send_normal_message "${CHAT[ID]}" "${msg}" 
 	fi
     else
 	send_normal_message "${USER[ID]}" "Sorry, you are not allowed to use this bot!"

examples/webhook/BASHBOT_HOME

@@ -0,0 +1,3 @@
+/usr/local/github/telegram-bot-bash-develop/DIST/telegram-bot-bash
+/usr/local/github/telegram-bot-bash-develop/STANDALONE
+/usr/local/telegram-bot-bash

examples/webhook/README.md

@@ -0,0 +1,100 @@
+#### [Examples](../README.md)
+
+## Bashbot webhook example
+
+### Webhook
+
+Bashbot default mode is to poll Telegram server for updates but Telegram offers webhook as a more efficient method to deliver updates.
+If your server is reachable from the Internet its possible to use the method described here.
+
+Prerequisite for receiving Telegram updates with webhook is a valid SSL certificate, a self signed certificate will not be sufficient.
+
+Webhook processing require special setup on server and Telegram side, therefore it's implemented as separate scripts and you need at least sudo rights to setup.
+
+#### Setup Apache webhook
+
+Prerequisite: An Apache webserver with a valid SLL certificate chain and php enabled.\
+This should work with other webservers also but it's not testet. 
+
+Setup webhook with Apache: 
+
+- install bashbot as described in [Bashbot Installation](../../doc/0_install.md)
+- create file `data-bot-bash/webhook-fifo-<botname>` (_\<botname\> as in `botconfig.jssh`_)
+- run `sudo bashbot.sh init` to setup bashbot to run as same user as web server (_e.g. www_)
+- create a directory in web root: `telegram/<your_bot_token>` (_<your_bot_token> as `botconfig.jssh`_)
+- give web server access to directory (_e.g.`chown www:www -R telegram`_)
+- go into the new directory and copy all files from `examples/webhook` to it
+- edit file `BASHBOT_HOME` to contain ithe Bashbot installation directory as first line (_other lines are ignored_)
+- execute `php index.php` with user id of web server to test write access to `data-bot-bash/webhook-fifo-<botname>
+
+Calling `https://<yourservername>/telegram/<your_bot_token>/` will execute `index.php`
+thus append received data to the file `data-bot-bash/webhook-fifo-<botname>`.
+E.g. `https://<yourservername>/telegram/<your_bot_token>/?json={"test":"me"}` will append `{"test":"me"}`.
+
+Now your Server is ready to receive updates from Telegram. 
+
+
+#### Default webhook processing
+
+This is the testet and supported default method for processing Telegram updates over webhook.
+
+To enable update processing delete the file `data-bot-bash/webhook-fifo-<botname>` if webhook is working as described above.
+Incoming Telegram updates are now forwarded to the script `bin/process_update.sh` for processing.
+
+On incoming Telegram updates the script is executed, it sources bashbot.sh and forward the update to Bashbot for processing.
+Even it seems overhead to source Bashbot for every update, it's more responsive and create less load than Bashbot polling mode.
+
+Nevertheles there are some limitations compared to polling mode:
+ - no startup actions
+ - `addons` and `TIMER_EVENTS` are not working
+
+Interactive and background jobs are working as of Bashbot Version 1.51.
+
+#### Full webhook processing
+
+Full webhook processing use an external script to imitate Bashbot polling mode with webhook.
+
+1. Default webook method must work first!
+2. run `bashbot.sh init` to setup bashbot to run with your user id
+2. Create a named pipe: `mkfifo data-bot-bash/webhook-fifo-botname` and give the web server write access to it
+3. execute `php index.php` with user id of web server to test write access to `data-bot-bash/webhook-fifo-<botname>
+4. Start script for Bashbot webhook polling mode:\
+`bin/process-batch.sh --startbot --watch data-bot-bash/webhook-fifo-<botname>`
+
+The script read updates from given file line by line and forward updates to Bashbot update processing. `--startbot` will run the startup actions
+(_e.g. load addons, start TIMER, trigger first run_) and `--watch` will wait for new updates instead of exit on end of file.
+Short form: 'bin/process-batch.sh -s -w'
+
+If script works as expected, you may run Bashbot webook polling in background by using `./bachbot.rc starthook/stophook`.
+
+To switch back to default processing delete fifo `data-bot-bash/webhook-fifo-<botname>` and stop `bin/process-batch.sh`.
+
+#### Enable webhook on Telegram side
+
+To get updates via webhook your server must be reachable from the internet and you must
+instruct Telegram where to deliver updates, this is done by calling bashbot function `set_webhook`.
+
+*Example:*
+
+```bash
+bin/any_command.sh set_webhook "https://myserver.com/telegram"
+```
+
+instruct Telegram to use the URL `https://myserver.com/telegram/<your_bot_token>/` to deliver updates.
+After you enable webhook to deliver Telegram updates it's no more possible to poll updates with `bashbot.sh start`.
+
+To stop delivering of Telegram updates via webhook run `bin/any_command.sh delete_webhook`.
+
+**Important**: Telegram will refuse to deliver updates if your webhook has no valid SSL certificate chain.
+
+
+#### Bash webhook
+
+A pure bash webhook implementation is not possible without extra software because Telegram delivers
+webhook updates only over secure TLS connections with a valid SSL certificate chain.
+
+`socat` looks like a tool to listen for Telegram updates from bash scripts, let's see ...
+
+
+#### $$VERSION$$ v1.52-1-g0dae2db
+

examples/webhook/index.php

@@ -0,0 +1,112 @@
+<?php
+/************************************************************
+ * @file        examples/webhook/index.php
+ * @description example webhook implementation for apache
+ *              write to fifo/file if writeable, else pipe to command 
+ *
+ *             first line of BASHBOT_HOME is used as bot home (optional)  
+ *             must start with /, not contain /. and min 20 characters long
+ *
+ * @author     KayM (gnadelwartz), kay@rrr.de
+ * @license    http://www.wtfpl.net/txt/copying/ WTFPLv2
+ * @since      30.01.2021 20:24
+ *
+#### $$VERSION$$ v1.52-1-g0dae2db
+ ***********************************************************/
+
+ // bashbot home dir
+ $CONFIG_HOME='BASHBOT_HOME';
+ $BASHBOT_HOME='/usr/local/telegram-bot-bash';
+ // read from config file
+ if (file_exists($CONFIG_HOME)) { 
+	$tmp = trim(fgets(fopen($CONFIG_HOME, 'r')));
+	// start with '/', not '/.', min 20 chars
+	if (substr($tmp,0,1) == '/' && strlen($tmp) >= 20 && strpos($tmp, '/.') === false) {
+		$BASHBOT_HOME=$tmp;
+	}
+ } 
+
+ // bashbot config file
+ $CONFIG=$BASHBOT_HOME.'/botconfig.jssh';
+ // set botname here or read botname from config file if unknown
+ $botname="unknown";
+ if ($botname == "unknown" && file_exists($CONFIG)) { 
+	$prefix='["botname"]	"';
+	$len=strlen($prefix);
+	$arr = file($CONFIG, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
+	foreach ($arr as $line) {
+		if(substr($line, 0, $len) == $prefix) {
+			$botname=substr($line, $len, strlen($line)-$len-1);
+		}
+	}
+
+ } 
+
+ // script endpoint
+ $cmd=$BASHBOT_HOME.'/bin/process_update.sh';
+ // default fifo endpoint
+ $fifo=$BASHBOT_HOME.'/data-bot-bash/webhook-fifo-'.$botname;
+
+ // prepeare read, e.g. run from CLI
+ $data='';
+ $input="php://input";
+ $json_file="json.txt";
+ if (php_sapi_name() == "cli") {
+	if(is_readable($json_file)) {
+		$input=$json_file;
+	} else {
+		$input="php://stdin";
+	}
+ }
+ // read request data
+ if($json = file_get_contents($input)) { 
+	$data = $json;
+ } else {
+	$data = implode(" ",$_POST);
+	if ($data == '') { $data = implode(" ",$_GET); }
+ }
+ // uncomment to save last received JSON
+ // file_put_contents($json_file, str_replace(array("\n", "\r"), '',$data). PHP_EOL));
+
+ // prepare for writing
+ if ($data == '') {
+	error_response(400, "No data received");
+ }
+ if (! chdir($BASHBOT_HOME)) {
+	error_response(403, "No route to bot home");
+ }
+
+ // fifo or command? 
+ if (! is_writeable($fifo)) {
+     // pipe to command
+     if (! file_exists($cmd)) {
+	error_response(502, "Webhook endpoint not found");
+     }
+     if (! $handle = popen( $cmd.' debug', 'w' )) {
+	error_response(503, "Can't open webhook command endpoint");
+     }
+ } else  {
+	// write to fifo
+	if (! $handle = fopen( $fifo, 'a' )) {
+		error_response(503, "Can't open webhook file endpoint");
+	}
+	flock($handle, LOCK_EX);
+ }
+ if (fwrite( $handle, str_replace(array("\n", "\r"), '',$data). PHP_EOL) === false) {
+	error_response(504, "Write to webhook failed");
+ }
+ flock($handle, LOCK_UN);
+ pclose($handle);
+/**/
+
+ function error_response($code, $msg) {
+    $api = substr(php_sapi_name(), 0, 3);
+    if ($api == 'cgi' || $api == 'fpm') {
+ 	header('Status: '.$code.' '.$msg);
+    } else {
+	$protocol = isset($_SERVER['SERVER_PROTOCOL']) ? $_SERVER['SERVER_PROTOCOL'] : 'HTTP/1.0';
+	header($protocol.' '.$code.' '.$msg);
+    }
+    exit('Error '.$code.': '.$msg. PHP_EOL);
+ }
+?>

examples/webhook/json.txt

@@ -0,0 +1 @@
+{"update_id":665220889,"message":{"message_id":760,"from":{"id":BOTADMIN,"is_bot":false,"first_name":"Kay","last_name":"M","username":"KayM","language_code":"de"},"chat":{"id":BOTADMIN,"first_name":"Kay","last_name":"M","username":"KayM","type":"private"},"date":1612029749,"text":"/info","entities":[{"offset":0,"length":5,"type":"bot_command"}]}}

modules/aliases.sh

@@ -1,11 +1,11 @@
 #!/bin/bash
-# file: modules/alaises.sh
+# file: modules/aliases.sh
 # do not edit, this file will be overwritten on update
 
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 #
 # will be automatically sourced from bashbot
 
@@ -23,36 +23,36 @@ _is_creator() {
 	user_is_creator "${CHAT[ID]}" "${USER[ID]}"
 }
 _is_allowed() {
-	user_is_allowed "${USER[ID]}" "${1}" "${CHAT[ID]}"
+	user_is_allowed "${USER[ID]}" "$1" "${CHAT[ID]}"
 }
 _leave() {
 	leave_chat "${CHAT[ID]}"
 }
 _kick_user() {
-	kick_chat_member "${CHAT[ID]}" "${1}"
+	kick_chat_member "${CHAT[ID]}" "$1"
 }
 _unban_user() {
-	unban_chat_member "${CHAT[ID]}" "${1}"
+	unban_chat_member "${CHAT[ID]}" "$1"
 }
 # easy sending of messages of messages
 _message() {
-	send_normal_message "${CHAT[ID]}" "${1}"
+	send_normal_message "${CHAT[ID]}" "$1"
 }
 _normal_message() {
-	send_normal_message "${CHAT[ID]}" "${1}"
+	send_normal_message "${CHAT[ID]}" "$1"
 }
 _html_message() {
-	send_html_message "${CHAT[ID]}" "${1}"
+	send_html_message "${CHAT[ID]}" "$1"
 }
 _markdown_message() {
-	send_markdown_message "${CHAT[ID]}" "${1}"
+	send_markdown_message "${CHAT[ID]}" "$1"
 }
 # easy handling of keyboards
 _inline_button() {
-	send_inline_button "${CHAT[ID]}" "" "${1}" "${2}" 
+	send_inline_button "${CHAT[ID]}" "" "$1" "$2" 
 }
 _inline_keyboard() {
-	send_inline_keyboard "${CHAT[ID]}" "" "${1}"
+	send_inline_keyboard "${CHAT[ID]}" "" "$1"
 }
 _keyboard_numpad() {
 	send_keyboard "${CHAT[ID]}" "" '["1","2","3"],["4","5","6"],["7","8","9"],["-","0","."]' "yes"

modules/answerInline.sh

@@ -5,20 +5,19 @@
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # will be automatically sourced from bashbot
 
 # source once magic, function named like file
 eval "$(basename "${BASH_SOURCE[0]}")(){ :; }"
 
-INLINE_QUERY=$URL'/answerInlineQuery'
 
 answer_inline_query() {
-	answer_inline_multi "${1}" "$(shift; inline_query_compose "$RANDOM" "$@")"
+	answer_inline_multi "$1" "$(shift; inline_query_compose "${RANDOM}" "$@")"
 }
 answer_inline_multi() {
-	sendJson "" '"inline_query_id": '"${1}"', "results": ['"${2}"']' "${INLINE_QUERY}"
+	sendJson "" '"inline_query_id": '"$1"', "results": ['"$2"']' "${URL}/answerInlineQuery"
 }
 
 # $1 unique ID for answer
@@ -27,76 +26,76 @@ answer_inline_multi() {
 # followed by the optional arguments: https://core.telegram.org/bots/api#inlinequeryresult
 inline_query_compose(){
 	local JSON="{}"
-	local ID="${1}"
+	local ID="$1"
 	local fours last
 								# title2Json title caption description markup inlinekeyboard
-	case "${2}" in
+	case "$2" in
 		# user provided media
-		"article"|"message") # article ID title message (markup description)
-			JSON='{"type":"article","id":"'$ID'","input_message_content": {"message_text":"'$4'"} '$(title2Json "$3" "" "$5" "$6" "$7")'}'
+		"article"|"message")	# article ID title message (markup description)
+			JSON='{"type":"article","id":"'${ID}'","input_message_content": {"message_text":"'$4'"} '$(title2Json "$3" "" "$5" "$6" "$7")'}'
 		;;
-		"photo") # photo ID photoURL (thumbURL title description caption)
+		"photo")	# photo ID photoURL (thumbURL title description caption)
 			[ -z "$4" ] && tumb="$3"
-			JSON='{"type":"photo","id":"'$ID'","photo_url":"'$3'","thumb_url":"'$4${tumb}'"'$(title2Json "$5" "$7" "$6" "$7" "$8")'}'
+			JSON='{"type":"photo","id":"'${ID}'","photo_url":"'$3'","thumb_url":"'$4${tumb}'"'$(title2Json "$5" "$7" "$6" "$7" "$8")'}'
 		;;
-		"gif") # gif ID photoURL (thumbURL title caption)
+		"gif")	# gif ID photoURL (thumbURL title caption)
 			[ -z "$4" ] && tumb="$3"
-			JSON='{"type":"gif","id":"'$ID'","gif_url":"'$3'", "thumb_url":"'$4${tumb}'"'$(title2Json "$5" "$6" "$7" "$8" "$9")'}'
+			JSON='{"type":"gif","id":"'${ID}'","gif_url":"'$3'", "thumb_url":"'$4${tumb}'"'$(title2Json "$5" "$6" "$7" "$8" "$9")'}'
 		;;
-		"mpeg4_gif") # mpeg4_gif ID mpegURL (thumbURL title caption)
+		"mpeg4_gif")	# mpeg4_gif ID mpegURL (thumbURL title caption)
 			[ -n "$4" ] && tumb='","thumb_url":"'$4'"'
-			JSON='{"type":"mpeg4_gif","id":"'$ID'","mpeg4_url":"'$3'"'${tumb}$(title2Json "$5" "$6" "" "$7" "$8")'}'
+			JSON='{"type":"mpeg4_gif","id":"'${ID}'","mpeg4_url":"'$3'"'${tumb}$(title2Json "$5" "$6" "" "$7" "$8")'}'
 		;;
-		"video") # video ID videoURL mime thumbURL title (caption)
-			JSON='{"type":"video","id":"'$ID'","video_url":"'$3'","mime_type":"'$4'","thumb_url":"'$5'"'$(title2Json "$6" "$7" "$8" "$9" "${10}")'}'
+		"video")	# video ID videoURL mime thumbURL title (caption)
+			JSON='{"type":"video","id":"'${ID}'","video_url":"'$3'","mime_type":"'$4'","thumb_url":"'$5'"'$(title2Json "$6" "$7" "$8" "$9" "${10}")'}'
 		;;
-		"audio") # audio ID audioURL title (caption)
-			JSON='{"type":"audio","id":"'$ID'","audio_url":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
+		"audio")	# audio ID audioURL title (caption)
+			JSON='{"type":"audio","id":"'${ID}'","audio_url":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
 		;;
-		"voice") # voice ID voiceURL title (caption)
-			JSON='{"type":"voice","id":"'$ID'","voice_url":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
+		"voice")	# voice ID voiceURL title (caption)
+			JSON='{"type":"voice","id":"'${ID}'","voice_url":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
 		;;
-		"document") # document ID title documentURL mimetype (caption description)
-			JSON='{"type":"document","id":"'$ID'","document_url":"'$4'","mime_type":"'$5'"'$(title2Json "$3" "$6" "$7" "$8" "$9")'}'
+		"document")	# document ID title documentURL mimetype (caption description)
+			JSON='{"type":"document","id":"'${ID}'","document_url":"'$4'","mime_type":"'$5'"'$(title2Json "$3" "$6" "$7" "$8" "$9")'}'
 		;;
-		"location") # location ID lat long title
-			JSON='{"type":"location","id":"'$ID'","latitude":"'$3'","longitude":"'$4'","title":"'$5'"}'
+		"location")	# location ID lat long title
+			JSON='{"type":"location","id":"'${ID}'","latitude":"'$3'","longitude":"'$4'","title":"'$5'"}'
 		;;
-		"venue") # venue ID lat long title (address forsquare)
+		"venue")	# venue ID lat long title (address forsquare)
 			[ -z "$6" ] && addr="$5"
 			[ -n "$7" ] && fours=',"foursquare_id":"'$7'"'
-			JSON='{"type":"venue","id":"'$ID'","latitude":"'$3'","longitude":"'$4'","title":"'$5'","address":"'$6${addr}'"'${fours}'}'
+			JSON='{"type":"venue","id":"'${ID}'","latitude":"'$3'","longitude":"'$4'","title":"'$5'","address":"'$6${addr}'"'${fours}'}'
 		;;
-		"contact") # contact ID phone first (last thumb)
+		"contact")	# contact ID phone first (last thumb)
 			[ -n "$5" ] && last=',"last_name":"'$5'"'
 			[ -n "$6" ] && tumb='","thumb_url":"'$6'"'
-			JSON='{"type":"contact","id":"'$ID'","phone_number":"'$3'","first_name":"'$4'"'${last}'"}'
+			JSON='{"type":"contact","id":"'${ID}'","phone_number":"'$3'","first_name":"'$4'"'${last}'"}'
 		;;
 								# title2Json title caption description markup inlinekeyboard
 		# Cached media stored in Telegram server
-		"cached_photo") # photo ID file (title description caption)
-			JSON='{"type":"photo","id":"'$ID'","photo_file_id":"'$3'"'$(title2Json "$4" "$6" "$5"  "$7" "$8")'}'
+		"cached_photo")	# photo ID file (title description caption)
+			JSON='{"type":"photo","id":"'${ID}'","photo_file_id":"'$3'"'$(title2Json "$4" "$6" "$5"  "$7" "$8")'}'
 		;;
-		"cached_gif") # gif ID file (title caption)
-			JSON='{"type":"gif","id":"'$ID'","gif_file_id":"'$3'"'$(title2Json "$4" "$5" "$6" "$7" "$8" )'}'
+		"cached_gif")	# gif ID file (title caption)
+			JSON='{"type":"gif","id":"'${ID}'","gif_file_id":"'$3'"'$(title2Json "$4" "$5" "$6" "$7" "$8" )'}'
 		;;
-		"cached_mpeg4_gif") # mpeg ID file (title caption)
-			JSON='{"type":"mpeg4_gif","id":"'$ID'","mpeg4_file_id":"'$3'"'$(title2Json "$4" "$5"  "" "$6" "$7")'}'
+		"cached_mpeg4_gif")	# mpeg ID file (title caption)
+			JSON='{"type":"mpeg4_gif","id":"'${ID}'","mpeg4_file_id":"'$3'"'$(title2Json "$4" "$5"  "" "$6" "$7")'}'
 		;;
-		"cached_sticker") # sticker ID file 
-			JSON='{"type":"sticker","id":"'$ID'","sticker_file_id":"'$3'"}'
+		"cached_sticker")	# sticker ID file 
+			JSON='{"type":"sticker","id":"'${ID}'","sticker_file_id":"'$3'"}'
 		;;
-		"cached_document") # document ID title file (description caption)
-			JSON='{"type":"document","id":"'$ID'","document_file_id":"'$4'"'$(title2Json "$3" "$6" "$5"  "$6" "$7")'}'
+		"cached_document")	# document ID title file (description caption)
+			JSON='{"type":"document","id":"'${ID}'","document_file_id":"'$4'"'$(title2Json "$3" "$6" "$5"  "$6" "$7")'}'
 		;;
-		"cached_video") # video ID file title (description caption)
-			JSON='{"type":"video","id":"'$ID'","video_file_id":"'$3'"'$(title2Json "$4" "$6" "$5" "$7" "$8")'}'
+		"cached_video")	# video ID file title (description caption)
+			JSON='{"type":"video","id":"'${ID}'","video_file_id":"'$3'"'$(title2Json "$4" "$6" "$5" "$7" "$8")'}'
 		;;
-		"cached_voice") # voice ID file title (caption)
-			JSON='{"type":"voice","id":"'$ID'","voice_file_id":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
+		"cached_voice")	# voice ID file title (caption)
+			JSON='{"type":"voice","id":"'${ID}'","voice_file_id":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
 		;;
-		"cached_audio") # audio ID file title (caption)
-			JSON='{"type":"audio","id":"'$ID'","audio_file_id":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
+		"cached_audio")	# audio ID file title (caption)
+			JSON='{"type":"audio","id":"'${ID}'","audio_file_id":"'$3'"'$(title2Json "$4" "$5" "" "" "$6")'}'
 		;;
 	esac
 

modules/background.sh

@@ -5,7 +5,8 @@
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+# shellcheck disable=SC1117,SC2059
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # will be automatically sourced from bashbot
 
@@ -45,12 +46,16 @@ start_back() {
 	printf '%s\n' "$1:$3:$2" >"${cmdfile}"
 	restart_back "$@"
 }
+# $1 chatid
+# $2 program
+# $3 jobname
+# $4 $5 parameters
 restart_back() {
 	local fifo; fifo="${DATADIR:-.}/$(procname "$1" "back-$3-")"
-	printf "%s: Start background job CHAT=%s JOB=%s CMD=%s\n" "$(date)" "${1}" "${fifo##*/}" "${2##*/} ${4} ${5}" >>"${UPDATELOG}"
+	log_update "Start background job CHAT=$1 JOB=${fifo##*/} CMD=${2##*/} $4 $5"
 	check_back "$1" "$3" && kill_proc "$1" "back-$3-"
-	nohup bash -c "{ $2 \"$4\" \"$5\" \"${fifo}\" | \"${SCRIPT}\" outproc \"${1}\" \"${fifo}\"; }" &>>"${fifo}.log" &
-	sleep 0.5 # give bg job some time to init
+	nohup bash -c "{ $2 \"$4\" \"$5\" \"${fifo}\" | \"${SCRIPT}\" outproc \"$1\" \"${fifo}\"; }" &>>"${fifo}.log" &
+	sleep 0.5	# give bg job some time to init
 }
 
 
@@ -61,10 +66,10 @@ start_proc() {
 	[ -z "$2" ] && return
 	[ -x "${2%% *}" ] || return 1
 	local fifo; fifo="${DATADIR:-.}/$(procname "$1")"
-	printf "%s: Start interacitve script CHAT=%s JOB=%s CMD=%s\n" "$(date)" "${1}" "${fifo##*/}" "${2} ${3} ${4}" >>"${UPDATELOG}"
 	check_proc "$1" && kill_proc "$1"
 	mkfifo "${fifo}"
-	nohup bash -c "{ $2 \"$4\" \"$5\" \"$fifo\" | \"${SCRIPT}\" outproc \"${1}\" \"${fifo}\"
+	log_update "Start interactive script CHAT=$1 JOB=${fifo##*/} CMD=$2 $3 $4"
+	nohup bash -c "{ $2 \"$4\" \"$5\" \"${fifo}\" | \"${SCRIPT}\" outproc \"$1\" \"${fifo}\"
 		rm \"${fifo}\"; [ -s \"${fifo}.log\" ] || rm -f \"${fifo}.log\"; }" &>>"${fifo}.log" &
 }
 
@@ -98,9 +103,11 @@ kill_proc() {
 	fifo="$(procname "$1" "$2")"
 	prid="$(proclist "${fifo}")"
 	fifo="${DATADIR:-.}/${fifo}"
-	printf "%s: Stop interacitve / background CHAT=%s JOB=%s\n" "$(date)" "${1}" "${fifo##*/}" >>"${UPDATELOG}"
 	# shellcheck disable=SC2086
-	[ -n "${prid}" ] && kill ${prid}
+	if [ -n "${prid}" ]; then
+		log_update "Stop interactive / background CHAT=$1 JOB=${fifo##*/}"
+		kill ${prid}
+	fi
 	[ -s "${fifo}.log" ] || rm -f "${fifo}.log"
 	[ -p "${fifo}" ] && rm -f "${fifo}";
 }
@@ -109,7 +116,7 @@ kill_proc() {
 # $2 message
 send_interactive() {
 	local fifo; fifo="${DATADIR:-.}/$(procname "$1")"
-	[ -p "${fifo}" ] && printf '%s\n' "$2" >"${fifo}" & # not blocking!
+	[ -p "${fifo}" ] && printf '%s\n' "$2" >"${fifo}" &	# not blocking!
 }
 
 # old style but may not work because of local checks
@@ -117,43 +124,42 @@ inproc() {
 	send_interactive "${CHAT[ID]}" "${MESSAGE[0]}"
 }
 
-# start stop all jobs
-# $1 command
-#	killb*
-#	suspendb*
-#	resumeb*
+# start stop all jobs 
+# $1 command #	kill suspend resume restart
 job_control() {
 	local BOT ADM content proc CHAT job fifo killall=""
 	BOT="$(getConfigKey "botname")"
-	ADM="$(getConfigKey "botadmin")"
-	debug_checks "Enter job_control" "${1}"
+	ADM="${BOTADMIN}"
+	debug_checks "Enter job_control" "$1"
+	# cleanup on start
+	[[ "$1" == "re"* ]] && bot_cleanup "startback"
 	for FILE in "${DATADIR:-.}/"*-back.cmd; do
-		[ "${FILE}" = "${DATADIR:-.}/*-back.cmd" ] && echo -e "${RED}No background processes.${NC}" && break
+		[ "${FILE}" = "${DATADIR:-.}/*-back.cmd" ] && printf "${RED}No background processes.${NN}" && break
 		content="$(< "${FILE}")"
 		CHAT="${content%%:*}"
 		job="${content#*:}"
 		proc="${job#*:}"
 		job="${job%:*}"
 		fifo="$(procname "${CHAT}" "${job}")" 
-		debug_checks "Execute job_control" "${1}" "${FILE##*/}"
+		debug_checks "Execute job_control" "$1" "${FILE##*/}"
 		case "$1" in
-		"resumeb"*|"backgr"*)
+		"resume"*|"restart"*)
 			printf "Restart Job: %s %s\n" "${proc}" " ${fifo##*/}"
 			restart_back "${CHAT}" "${proc}" "${job}"
 			# inform botadmin about stop
 			[ -n "${ADM}" ] && send_normal_message "${ADM}" "Bot ${BOT} restart background jobs ..." &
 			;;
-		"suspendb"*)
+		"suspend"*)
 			printf "Suspend Job: %s %s\n" "${proc}" " ${fifo##*/}"
 			kill_proc "${CHAT}" "${job}"
 			# inform botadmin about stop
 			[ -n "${ADM}" ] && send_normal_message "${ADM}" "Bot ${BOT} suspend background jobs ..." &
 			killall="y"
 			;;
-		"killb"*)
+		"kill"*)
 			printf "Kill Job: %s %s\n" "${proc}" " ${fifo##*/}"
 			kill_proc "${CHAT}" "${job}"
-			rm -f "${FILE}" # remove job
+			rm -f "${FILE}"	# remove job
 			# inform botadmin about stop
 			[ -n "${ADM}" ] && send_normal_message "${ADM}" "Bot ${BOT} kill  background jobs ..." &
 			killall="y"
@@ -162,7 +168,7 @@ job_control() {
 		# send message only onnfirst job
 		ADM=""
 	done
-	debug_checks "end job_control" "${1}"
+	debug_checks "end job_control" "$1"
 	# kill all requestet. kill ALL background jobs, even not listed in data-bot-bash
 	[ "${killall}" = "y" ] && killallproc "back-"
 }

modules/chatMember.sh

@@ -5,35 +5,122 @@
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.2-dev-14-g6ec00d4
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # will be automatically sourced from bashbot
 
 # source once magic, function named like file
 eval "$(basename "${BASH_SOURCE[0]}")(){ :; }"
 
-LEAVE_URL=$URL'/leaveChat'
-KICK_URL=$URL'/kickChatMember'
-UNBAN_URL=$URL'/unbanChatMember'
-GETMEMBER_URL=$URL'/getChatMember'
 
-# usage: status="$(get_chat_member_status "chat" "user")"
-get_chat_member_status() {
-	sendJson "$1" '"user_id":'"$2"'' "$GETMEMBER_URL"
-	# shellcheck disable=SC2154
-	JsonGetString '"result","status"' <<< "$res"
+# manage chat functions -------
+# $1 chat 
+new_chat_invite() {
+	sendJson "$1" "" "${URL}/exportChatInviteLink"
+	[ "${BOTSENT[OK]}" = "true" ] && printf "%s\n" "${BOTSENT[RESULT]}"
+}
+
+# $1 chat, $2 user_id, $3 title 
+set_chatadmin_title() {
+	sendJson "$1" '"user_id":'"$2"',"custom_title": "'"$3"'"' "${URL}/setChatAdministratorCustomTitle"
+}
+# $1 chat, $2 title 
+set_chat_title() {
+	sendJson "$1" '"title": "'"$2"'"' "${URL}/setChatTitle"
+}
+
+# $1 chat, $2 title 
+set_chat_description() {
+	sendJson "$1" '"description": "'"$2"'"' "${URL}/setChatDescription"
+}
+
+# $1 chat  $2 file
+set_chat_photo() {
+	local file; file="$(checkUploadFile "$1" "$2" "set_chat_photo")"
+	[ -z "${file}" ] && return 1
+	sendUpload "$1" "photo" "${file}" "${URL}/setChatPhoto" 
+}
+# $1 chat 
+delete_chat_photo() {
+	sendJson "$1" "" "${URL}/deleteChatPhoto"
+}
+
+# $1 chat, $2 message_id 
+pin_chat_message() {
+	sendJson "$1" '"message_id": "'"$2"'"' "${URL}/pinChatMessage"
+}
+
+# $1 chat, $2 message_id 
+unpin_chat_message() {
+	sendJson "$1" '"message_id": "'"$2"'"' "${URL}/unpinChatMessage"
+}
+
+# $1 chat 
+unpinall_chat_message() {
+	sendJson "$1" "" "${URL}/unpinAllChatMessages"
+}
+
+# $1 chat 
+delete_chat_stickers() {
+	sendJson "$1" "" "${URL}/deleteChatStickerSet"
+}
+
+# manage chat member functions -------
+# $1 chat 
+chat_member_count() {
+	sendJson "$1" "" "${URL}/getChatMembersCount"
+	[ "${BOTSENT[OK]}" = "true" ] && printf "%s\n" "${BOTSENT[RESULT]}"
 }
 
 kick_chat_member() {
-	sendJson "$1" 'user_id: '"$2"'' "$KICK_URL"
+	sendJson "$1" '"user_id": '"$2"'' "${URL}/kickChatMember"
 }
 
 unban_chat_member() {
-	sendJson "$1" 'user_id: '"$2"'' "$UNBAN_URL"
+	sendJson "$1" '"user_id": '"$2"'' "${URL}/unbanChatMember"
 }
 
 leave_chat() {
-	sendJson "$1" "" "$LEAVE_URL"
+	sendJson "$1" "" "${URL}/leaveChat"
+}
+
+# $1 chat, $2 userid, $3 ... "right[:true]" default false
+# right:  is_anonymous change_info post_messages edit_messages delete_messages invite_users restrict_members pin_messages promote_member
+promote_chat_member() {
+	local arg bool json chat="$1" user="$2; shift 2"
+	for arg in "$@"
+	do
+		# default false
+		bool=false; [ "${arg##*:}" = "true" ] && bool="true"
+		# expand args
+		case "${arg}" in
+			*"anon"*)	arg="is_anonymous";;
+			*"change"*)	arg="can_change_info";;
+			*"post"*)	arg="can_post_messages";;
+			*"edit"*)	arg="can_edit_messages";;
+			*"delete"*)	arg="can_delete_messages";;
+			*"pin"*)	arg="can_pin_messages";;
+			*"invite"*)	arg="can_invite_users";;
+			*"restrict"*)	arg="can_restrict_members";;
+			*"promote"*)	arg="can_promote_members";;
+			*) 	[ -n "${BASHBOTDEBUG}" ] && log_debug "promote_chat_member: unknown promotion CHAT=${chat} USER=${user} PROM=${arg}"
+				continue;; 
+		esac
+		# compose json
+		[ -n "${json}" ] && json+=","
+		json+='"'"${arg}"'": "'"${bool}"'"'
+	done
+	sendJson "${chat}" '"user_id":'"${user}"','"${json}"'' "${URL}/promoteChatMember"
+}
+
+# bashbot specific functions ---------
+
+# usage: status="$(get_chat_member_status "chat" "user")"
+# $1 chat # $2 user
+get_chat_member_status() {
+	sendJson "$1" '"user_id":'"$2"'' "${URL}/getChatMember"
+	# shellcheck disable=SC2154
+	printf "%s\n" "${UPD["result,status"]}"
 }
 
 user_is_creator() {
@@ -42,7 +129,14 @@ user_is_creator() {
 	return 1 
 }
 
+# $1 chat
+bot_is_admin() {
+	user_is_admin "$1" "$(getConfigKey "botid")"
+}
+
+# $1 chat # $2 user
 user_is_admin() {
+	[[ -z "$1" || -z "$2" ]] && return 1
 	[ "${1:--}" == "${2:-+}" ] && return 0
 	user_is_botadmin "$2" && return 0
 	local me; me="$(get_chat_member_status "$1" "$2")"
@@ -50,27 +144,30 @@ user_is_admin() {
 	return 1 
 }
 
+# $1 user
 user_is_botadmin() {
-	local admin; admin="$(getConfigKey "botadmin")"; [ -z "${admin}" ] && return 1
-	[[ "${admin}" == "${1}" || "${admin}" == "${2}" ]] && return 0
-	#[[ "${admin}" = "@*" ]] && [[ "${admin}" = "${2}" ]] && return 0
-	if [ "${admin}" = "?" ]; then setConfigKey "botadmin" "${1:-?}"; return 0; fi
+	[ -z "$1" ] && return 1
+	[ -z "${BOTADMIN}" ] && return 1
+	[[ "${BOTADMIN}" == "$1" || "${BOTADMIN}" == "$2" ]] && return 0
+	if [ "${BOTADMIN}" = "?" ]; then setConfigKey "botadmin" "${1:-?}"; BOTADMIN="${1:-?}"; return 0; fi
 	return 1
 }
 
+# $1 user # $2 key # $3 chat
 user_is_allowed() {
 	[ -z "$1" ] && return 1
+	user_is_admin "$1" && return 0
 	# user can do everything
-	grep -F -xq "$1:*:*" <"${BOTACL}" && return 0
+	grep -F -xq "$1:*:*" "${BOTACL}" && return 0
 	[ -z "$2" ] && return 1
 	# user is allowed todo one action in every chat
-	grep -F -xq "$1:$2:*" <"${BOTACL}" && return 0
+	grep -F -xq "$1:$2:*" "${BOTACL}" && return 0
 	# all users are allowed to do one action in every chat
-	grep -F -xq "ALL:$2:*" <"${BOTACL}" && return 0
+	grep -F -xq "ALL:$2:*" "${BOTACL}" && return 0
 	[ -z "$3" ] && return 1
 	# user is allowed to do one action in one chat
-	grep -F -xq "$1:$2:$3" <"${BOTACL}" && return 0
+	grep -F -xq "$1:$2:$3" "${BOTACL}" && return 0
 	# all users are allowed to do one action in one chat
-	grep -F -xq "ALL:$2:$3" <"${BOTACL}" && return 0
+	grep -F -xq "ALL:$2:$3" "${BOTACL}" && return 0
 	return 1
 }

modules/jsonDB.sh

@@ -5,7 +5,7 @@
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
 #
 # source from commands.sh to use jsonDB functions
 #
@@ -25,7 +25,31 @@ eval "$(basename "${BASH_SOURCE[0]}")(){ :; }"
 # tinybox
 
 # lockfile filename.flock is persistent and will be testet with flock for active lock (file open)
-export JSSH_LOCKNAME=".flock"
+export JSSHDB_LOCKNAME=".flock"
+# an array value containing this string will not saveed to DB (unset)
+export JSSHDB_UNSET="99999999999999999999_JSSHDB_UNSET_99999999999999999999"
+
+# in UTF-8 äöü etc. are part of [:alnum:] and ranges (e.g. a-z), but we want ASCII a-z ranges!
+# for more information see  doc/4_expert.md#Character_classes
+azazaz='abcdefghijklmnopqrstuvwxyz'	# a-z   :lower:
+AZAZAZ='ABCDEFGHIJKLMNOPQRSTUVWXYZ'	# A-Z   :upper:
+o9o9o9='0123456789'			# 0-9   :digit:
+azAZaz="${azazaz}${AZAZAZ}"	# a-zA-Z	:alpha:
+azAZo9="${azAZaz}${o9o9o9}"	# a-zA-z0-9	:alnum:
+
+# characters allowed for key in key/value pairs
+JSSH_KEYOK="[-${azAZo9},._]"
+
+# read string from stdin and and strip invalid characters
+# $1 - invalid charcaters are replaced with first character
+#      or deleted if $1 is empty
+jssh_stripKey() {	# tr: we must escape first - in [-a-z...]
+	if [[ "$1" =~ ^${JSSH_KEYOK} ]]; then	# tr needs [\-...
+ 		tr -c "${JSSH_KEYOK/\[-/[\\-}\r\n" "${1:0:1}"
+	else
+ 		tr -dc "${JSSH_KEYOK/\[-/[\\-}\r\n"
+	fi
+}
 
 # use flock if command exist
 if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
@@ -42,7 +66,7 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
 	[ -z "${DB}" ] && return 1
 	[ ! -f "${DB}" ] && return 2
 	# shared lock, many processes can read, max wait 1s
-	{ flock -s -w 1 200; Json2Array "$1" <"${DB}"; } 200>"${DB}${JSSH_LOCKNAME}"
+	{ flock -s -w 1 200; Json2Array "$1" <"${DB}"; } 200>"${DB}${JSSHDB_LOCKNAME}"
   }
 
   # write ARRAY content to a file in JSON.sh format
@@ -54,7 +78,7 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
 	[ -z "${DB}" ] && return 1
 	[ ! -f "${DB}" ] && return 2
 	# exclusive lock, no other process can read or write, maximum wait to get lock is 10s
-	{ flock -e -w 10 200; Array2Json "$1" >"${DB}"; } 200>"${DB}${JSSH_LOCKNAME}"
+	{ flock -e -w 10 200; Array2Json "$1" >"${DB}"; } 200>"${DB}${JSSHDB_LOCKNAME}"
   }
 
   # update/write ARRAY content in file without deleting keys not in ARRAY
@@ -63,20 +87,20 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
   # complex slow, warpper async
   jssh_updateDB() {
 	# for atomic update we can't use read/writeDB
-	[ -z "${2}" ] && return 1
-	local DB="${2}.jssh" # check in async
+	[ -z "$2" ] && return 1
+	local DB="$2.jssh"	# check in async
 	[ ! -f "${DB}" ] && return 2
-	{ flock -e -w 10 200; jssh_updateDB_async "$@"; } 200>"${DB}${JSSH_LOCKNAME}"
+	{ flock -e -w 10 200; jssh_updateDB_async "$@"; } 200>"${DB}${JSSHDB_LOCKNAME}"
   }
 
   # insert, update, apped key/value to jsshDB
   # $1 key name, can only contain -a-zA-Z0-9,._
   # $2 key value
   # $3 filename (must exist!), must be relative to BASHBOT_ETC, and not contain '..'
-  alias jssh_insertDB=jssh_insertKeyDB # backward compatibility
+  alias jssh_insertDB=jssh_insertKeyDB	# backward compatibility
   # renamed to be more consistent
   jssh_insertKeyDB() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
 	local DB; DB="$(jssh_checkDB "$3")"
 	[ -z "${DB}" ] && return 1
 	[ ! -f "${DB}" ] && return 2
@@ -84,7 +108,7 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
 	{ flock -e -w 2 200
 	 # it's append, but last one counts, its a simple DB ...
 	  printf '["%s"]\t"%s"\n' "${1//,/\",\"}" "${2//\"/\\\"}" >>"${DB}"
-	} 200>"${DB}${JSSH_LOCKNAME}"
+	} 200>"${DB}${JSSHDB_LOCKNAME}"
 	
   }
 
@@ -93,11 +117,11 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
   # $2 filename (must exist!), must be relative to BASHBOT_ETC, and not contain '..'
   # medium complex slow, wrapper async
   jssh_deleteKeyDB() {
-	[ -z "${2}" ] && return 1
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
-	local DB="${2}.jssh"
+	[ -z "$2" ] && return 1
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
+	local DB="$2.jssh"
 	# start atomic delete here, exclusive max wait 10s 
-	{ flock -e -w 10 200; jssh_deleteKeyDB_async "$@"; } 200>"${DB}${JSSH_LOCKNAME}"
+	{ flock -e -w 10 200; jssh_deleteKeyDB_async "$@"; } 200>"${DB}${JSSHDB_LOCKNAME}"
   }
 
   # get key/value from jsshDB
@@ -105,13 +129,13 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
   # $2 filename (must exist!), must be relative to BASHBOT_ETC, and not contain '..'
   alias jssh_getDB=jssh_getKeyDB
   jssh_getKeyDB() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
 	local DB; DB="$(jssh_checkDB "$2")"
 	[ -z "${DB}" ] && return 1
 	# start atomic delete here, exclusive max wait 1s 
 	{ flock -s -w 1 200
-	[ -r "${DB}" ] && sed -n 's/\["'"$1"'"\]\t*"\(.*\)"/\1/p' <"${DB}" | tail -n 1
-	} 200>"${DB}${JSSH_LOCKNAME}"
+	[ -r "${DB}" ] && sed -n 's/\["'"$1"'"\]\t*"\(.*\)"/\1/p' "${DB}" | tail -n 1
+	} 200>"${DB}${JSSHDB_LOCKNAME}"
   }
 
 
@@ -122,11 +146,11 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
   # side effect: if $3 is not given, we add to end of file to be as fast as possible
   # complex, wrapper to async
   jssh_countKeyDB() {
-	[ -z "${2}" ] && return 1
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
-	local DB="${2}.jssh"
+	[ -z "$2" ] && return 1
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
+	local DB="$2.jssh"
 	# start atomic delete here, exclusive max wait 5 
-	{ flock -e -w 5 200; jssh_countKeyDB_async "$@"; } 200>"${DB}${JSSH_LOCKNAME}"
+	{ flock -e -w 5 200; jssh_countKeyDB_async "$@"; } 200>"${DB}${JSSHDB_LOCKNAME}"
   }
 
   # update key/value in place to jsshDB
@@ -135,19 +159,19 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
   # $3 filename (must exist!), must be relative to BASHBOT_ETC, and not contain '..'
   #no own locking, so async is the same as updatekeyDB
   jssh_updateKeyDB() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
-	[ -z "${3}" ] && return 1
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
+	[ -z "$3" ] && return 1
 	declare -A updARR
 	# shellcheck disable=SC2034
 	updARR["$1"]="$2"
-	jssh_updateDB "updARR" "${3}" || return 3
+	jssh_updateDB "updARR" "$3" || return 3
   }
 
   # $1 filename (must exist!), must be relative to BASHBOT_ETC, and not contain '..'
   jssh_clearDB() {
 	local DB; DB="$(jssh_checkDB "$1")"
 	[ -z "${DB}" ] && return 1
-	{ flock -e -w 10 200; printf '' >"${DB}"; } 200>"${DB}${JSSH_LOCKNAME}"
+	{ flock -e -w 10 200; printf '' >"${DB}"; } 200>"${DB}${JSSHDB_LOCKNAME}"
   } 
 
   # updates Array if DB file has changed since last call
@@ -156,11 +180,11 @@ if [ "$(LC_ALL=C type -t "flock")" = "file" ]; then
   # $3 id used to identify caller
   # medium complex, wrapper async
   jssh_updateArray() { 
-	[ -z "${2}" ] && return 1
-	local DB="${2}.jssh" # name check in async
+	[ -z "$2" ] && return 1
+	local DB="$2.jssh"	# name check in async
 	[ ! -f "${DB}" ] && return 2
 	declare -n ARRAY="$1"
-	[[ -z "${ARRAY[*]}" ||  "${DB}" -nt "${DB}.last${3}" ]] && touch "${DB}.last${3}" && jssh_readDB "${1}" "${2}"
+	[[ -z "${ARRAY[*]}" ||  "${DB}" -nt "${DB}.last$3" ]] && touch "${DB}.last$3" && jssh_readDB "$1" "$2"
   }
 
 else
@@ -194,7 +218,7 @@ jssh_newDB_async() { jssh_newDB "$@"; }
 jssh_newDB() {
 	local DB; DB="$(jssh_checkDB "$1")"
 	[ -z "${DB}" ] && return 1
-	[ -f "${DB}" ] && return 2 # already exist
+	[ -f "${DB}" ] && return 2	# already exist
 	touch "${DB}"
 } 
 
@@ -204,11 +228,11 @@ jssh_checkDB_async() { jssh_checkDB "$@"; }
 jssh_checkDB(){
 	local DB
 	[ -z "$1" ] && return 1
-	[[ "$1" = *'..'* ]] && return 2
+	[[ "$1" = *'../.'* ]] && return 2
 	if [[ "$1" == "${BASHBOT_VAR:-.}"* ]] || [[ "$1" == "${BASHBOT_DATA:-.}"* ]]; then
-		DB="${1}.jssh"
+		DB="$1.jssh"
 	else
-		DB="${BASHBOT_VAR:-.}/${1}.jssh"
+		DB="${BASHBOT_VAR:-.}/$1.jssh"
 	fi
 	[ "${DB}" != ".jssh" ] && printf '%s' "${DB}"
 }
@@ -232,7 +256,7 @@ jssh_writeDB_async() {
 }
 
 jssh_updateDB_async() {
-	[ -z "${2}" ] && return 1
+	[ -z "$2" ] && return 1
 	declare -n ARRAY="$1"
 	[ -z "${ARRAY[*]}" ] && return 1
 	declare -A oldARR
@@ -253,7 +277,7 @@ jssh_updateDB_async() {
 
 jssh_insertDB_async() { jssh_insertKeyDB "$@"; }
 jssh_insertKeyDB_async() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
 	local DB; DB="$(jssh_checkDB "$3")"
 	[ -z "${DB}" ] && return 1
 	[ ! -f "${DB}" ] && return 2
@@ -263,7 +287,7 @@ jssh_insertKeyDB_async() {
 }
 
 jssh_deleteKeyDB_async() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
 	local DB; DB="$(jssh_checkDB "$2")"
 	[ -z "${DB}" ] && return 1
 	declare -A oldARR
@@ -273,14 +297,14 @@ jssh_deleteKeyDB_async() {
 }
 
 jssh_getKeyDB_async() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
 	local DB; DB="$(jssh_checkDB "$2")"
 	[ -z "${DB}" ] && return 1
-	[ -r "${DB}" ] && sed -n 's/\["'"$1"'"\]\t*"\(.*\)"/\1/p' <"${DB}" | tail -n 1
+	[ -r "${DB}" ] && sed -n 's/\["'"$1"'"\]\t*"\(.*\)"/\1/p' "${DB}" | tail -n 1
 }
 
 jssh_countKeyDB_async() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
 	local VAL DB; DB="$(jssh_checkDB "$2")"
 	[ -z "${DB}" ] && return 1
 	# start atomic delete here, exclusive max wait 5 
@@ -291,23 +315,23 @@ jssh_countKeyDB_async() {
 		Array2Json  "oldARR" >"${DB}"
 	elif [ -r "${DB}" ]; then
 		# it's append, but last one counts, its a simple DB ...
-		VAL="$(sed -n 's/\["'"$1"'"\]\t*"\(.*\)"/\1/p' <"${DB}" | tail -n 1)"
+		VAL="$(sed -n 's/\["'"$1"'"\]\t*"\(.*\)"/\1/p' "${DB}" | tail -n 1)"
 		printf '["%s"]\t"%s"\n' "${1//,/\",\"}" "$((++VAL))" >>"${DB}"
 	fi
   }
 
-# updatie key/value in place to jsshDB
+# update key/value in place to jsshDB
 # $1 key name, can only contain -a-zA-Z0-9,._
 # $2 key value
 # $3 filename (must exist!), must be relative to BASHBOT_ETC, and not contain '..'
 #no own locking, so async is the same as updatekeyDB
 jssh_updateKeyDB_async() {
-	[[ "$1" =~ ^[-a-zA-Z0-9,._]+$ ]] || return 3
-	[ -z "${3}" ] && return 1
+	[[ "$1" =~ ^${JSSH_KEYOK}+$ ]] || return 3
+	[ -z "$3" ] && return 1
 	declare -A updARR
 	# shellcheck disable=SC2034
 	updARR["$1"]="$2"
-	jssh_updateDB_async "updARR" "${3}" || return 3
+	jssh_updateDB_async "updARR" "$3" || return 3
 }
 
 jssh_clearDB_async() {
@@ -321,7 +345,7 @@ function jssh_updateArray_async() {
 	[ -z "${DB}" ] && return 1
 	[ ! -f "${DB}" ] && return 2
 	declare -n ARRAY="$1"
-	[[ -z "${ARRAY[*]}" ||  "${DB}" -nt "${DB}.last${3}" ]] && touch "${DB}.last${3}" && jssh_readDB_async "${1}" "${2}"
+	[[ -z "${ARRAY[*]}" ||  "${DB}" -nt "${DB}.last$3" ]] && touch "${DB}.last$3" && jssh_readDB_async "$1" "$2"
 }
 
 ##############
@@ -332,20 +356,22 @@ function jssh_updateArray_async() {
 # $1 ARRAY name, must be declared with "declare -A ARRAY" before calling
 Json2Array() {
 	# shellcheck disable=SC1091,SC1090
-	[ -z "$1" ] || source <( printf "$1"'=( %s )' "$(sed -E -n -e ':x /"text"\]/ { N; s/([^"])\n/\1\\n/g ; tx }' -e '/\["[-0-9a-zA-Z_,."]+"\]\+*\t/ s/\t/=/gp' -e 's/=(true|false)/="\1"/')" )
+	# step 1: output only basic pattern
+	[ -z "$1" ] || source <( printf "$1"'=( %s )'\
+		 "$(sed -E -n -e 's/[`´]//g' -e 's/\t(true|false)/\t"\1"/' -e 's/([^\]|^)\$/\1\\$/g' -e '/\["[-0-9a-zA-Z_,."]+"\]\+*\t/ s/\t/=/p')" )
 }
 # get Config Key from jssh file without jsshDB
 # output ARRAY as JSON.sh style data
 # $1 ARRAY name, must be declared with "declare -A ARRAY" before calling
 Array2Json() {
 	[ -z "$1" ] && return 1
-	local key val
+	local key
 	declare -n ARRAY="$1"
 	for key in "${!ARRAY[@]}"
        	do
-		# in case val contains newline convert to \n
-		val="${ARRAY[${key}]//$'\n'/\\n}"
-		printf '["%s"]\t"%s"\n' "${key//,/\",\"}" "${val//\"/\\\"}"
+		[[ ! "${key}" =~ ^${JSSH_KEYOK}+$ || "${ARRAY[${key}]}" == "${JSSHDB_UNSET}" ]] && continue
+		# in case value contains newline convert to \n
+		: "${ARRAY[${key}]//$'\n'/\\n}"
+		printf '["%s"]\t"%s"\n' "${key//,/\",\"}" "${_//\"/\\\"}"
        	done
 }
-

modules/processUpdates.sh

@@ -0,0 +1,507 @@
+#!/bin/bash
+##################################################################
+#
+# File: processUpdates.sh 
+# Note: DO NOT EDIT! this file will be overwritten on update
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+##################################################################
+
+##############
+# manage webhooks
+
+# $1 URL to sed updates to: https://host.dom[:port][/path], port and path are optional
+#      port must be 443, 80, 88 8443, TOKEN will be added to URL for security
+#      e.g. https://myhost.com -> https://myhost.com/12345678:azndfhbgdfbbbdsfg
+# $2 max connections 1-100 default 1 (because of bash ;-)
+set_webhook() {
+	local  url='"url": "'"$1/${BOTTOKEN}/"'"'
+	local  max=',"max_connections": 1'
+	[[ "$2" =~ ^[0-9]+$ ]] && max=',"max_connections": '"$2"''
+	# shellcheck disable=SC2153
+	sendJson "" "${url}${max}" "${URL}/setWebhook"
+	unset "BOTSENT[ID]" "BOTSENT[CHAT]"
+
+}
+
+get_webhook_info() {
+	sendJson "" "" "${URL}/getWebhookInfo"
+	if [ "${BOTSENT[OK]}" = "true" ]; then
+		BOTSENT[URL]="${UPD[result,url]}"
+		BOTSENT[COUNT]="${UPD[result,pending_update_count]}"
+		BOTSENT[CERT]="${UPD[result,has_custom_certificate]}"
+		BOTSENT[LASTERR]="${UPD[result,last_error_message]}"
+		unset "BOTSENT[ID]" "BOTSENT[CHAT]"
+	fi
+}
+
+# $1 drop pending updates true/false, default false
+delete_webhook() {
+	local drop; [ "$1" = "true" ] && drop='"drop_pending_updates": true'
+	sendJson "" "${drop}" "${URL}/deleteWebhook"
+	unset "BOTSENT[ID]" "BOTSENT[CHAT]"
+}
+
+################
+# processing of array of updates starts here
+process_multi_updates() {
+	local max num debug="$1"
+	# get num array elements
+	max="$(grep -F ',"update_id"]'  <<< "${UPDATE}" | tail -1 | cut -d , -f 2 )"
+	# escape bash $ expansion bug
+	UPDATE="${UPDATE//$/\\$}"
+	# convert updates to bash array
+	Json2Array 'UPD' <<<"${UPDATE}"
+	# iterate over array
+	for ((num=0; num<=max; num++)); do
+		process_update "${num}" "${debug}"
+	done
+}
+
+################
+# processing of a single array item of update
+# $1 array index
+process_update() {
+	local chatuser="Chat" num="$1" debug="$2" 
+	pre_process_message "${num}"
+	# log message on debug
+	[[ -n "${debug}" ]] && log_message "New Message ==========\n$(grep -F '["result",'"${num}" <<<"${UPDATE}")"
+
+	# check for users / groups to ignore, inform them ...
+	jssh_updateArray_async "BASHBOTBLOCKED" "${BLOCKEDFILE}"
+	if [ -n "${USER[ID]}" ] && [[ -n "${BASHBOTBLOCKED[${USER[ID]}]}" || -n "${BASHBOTBLOCKED[${CHAT[ID]}]}" ]];then
+		[  -n "${BASHBOTBLOCKED[${USER[ID]}]}" ] && chatuser="User"
+		[ "${NOTIFY_BLOCKED_USERS}" == "yes" ] &&\
+			send_normal_message "${CHAT[ID]}" "${chatuser} blocked because: ${BASHBOTBLOCKED[${USER[ID]}]} ${BASHBOTBLOCKED[${CHAT[ID]}]}" &
+		return
+	fi
+
+	# process per message type
+	if [ -n "${iQUERY[ID]}" ]; then
+		process_inline_query "${num}" "${debug}"
+	        printf "%(%c)T: Inline Query update received FROM=%s iQUERY=%s\n" -1\
+			"${iQUERY[USERNAME]:0:20} (${iQUERY[USER_ID]})" "${iQUERY[0]}" >>"${UPDATELOG}"
+	elif [ -n "${iBUTTON[ID]}" ]; then
+		process_inline_button "${num}" "${debug}"
+	        printf "%(%c)T: Inline Button update received FROM=%s CHAT=%s CALLBACK=%s DATA:%s \n" -1\
+			"${iBUTTON[USERNAME]:0:20} (${iBUTTON[USER_ID]})" "${iBUTTON[CHAT_ID]}" "${iBUTTON[ID]}" "${iBUTTON[DATA]}" >>"${UPDATELOG}"
+	else
+		if grep -qs -e '\["result",'"${num}"',"edited_message"' <<<"${UPDATE}"; then
+			# edited message
+			UPDATE="${UPDATE//,${num},\"edited_message\",/,${num},\"message\",}"
+			Json2Array 'UPD' <<<"${UPDATE}"
+			MESSAGE[0]="/_edited_message "
+		fi
+		process_message "${num}" "${debug}"
+	        printf "%(%c)T: update received FROM=%s CHAT=%s CMD=%s\n" -1 "${USER[USERNAME]:0:20} (${USER[ID]})"\
+			"${CHAT[USERNAME]:0:20}${CHAT[TITLE]:0:30} (${CHAT[ID]})"\
+			"${MESSAGE:0:30}${CAPTION:0:30}${URLS[*]}" >>"${UPDATELOG}"
+		if [[ -z "${USER[ID]}" || -z "${CHAT[ID]}" ]]; then
+			printf "%(%c)T: IGNORE unknown update type: %s\n" -1 "$(grep '\["result",'"${num}"'.*,"id"\]' <<<"${UPDATE}")" >>"${UPDATELOG}"
+			return 1
+		fi
+	fi
+	#####
+	# process inline and message events
+	# first classic command dispatcher
+	# shellcheck disable=SC2153,SC1090
+	{ source "${COMMANDS}" "${debug}"; } &
+
+	# then all registered addons
+	if [ -z "${iQUERY[ID]}" ]; then
+		event_message "${debug}"
+	else
+		event_inline "${debug}"
+	fi
+
+	# last count users
+	jssh_countKeyDB_async "${CHAT[ID]}" "${COUNTFILE}"
+}
+
+pre_process_message(){
+	local num="$1"
+	# unset everything to not have old values
+	CMD=( ); iQUERY=( ); iBUTTON=(); MESSAGE=(); CHAT=(); USER=(); CONTACT=(); LOCATION=(); unset CAPTION
+	REPLYTO=( ); FORWARD=( ); URLS=(); VENUE=( ); SERVICE=( ); NEWMEMBER=( ); LEFTMEMBER=( ); PINNED=( ); MIGRATE=( )
+	iQUERY[ID]="${UPD["result,${num},inline_query,id"]}"
+	iBUTTON[ID]="${UPD["result,${num},callback_query,id"]}"
+	CHAT[ID]="${UPD["result,${num},message,chat,id"]}"
+	USER[ID]="${UPD["result,${num},message,from,id"]}"
+	[ -z "${CHAT[ID]}" ] && CHAT[ID]="${UPD["result,${num},edited_message,chat,id"]}"
+	[ -z "${USER[ID]}" ] && USER[ID]="${UPD["result,${num},edited_message,from,id"]}"
+	# always true
+	return 0
+}
+
+process_inline_query() {
+	local num="$1"
+	iQUERY[0]="$(JsonDecode "${UPD["result,${num},inline_query,query"]}")"
+	iQUERY[USER_ID]="${UPD["result,${num},inline_query,from,id"]}"
+	iQUERY[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},inline_query,from,first_name"]}")"
+	iQUERY[LAST_NAME]="$(JsonDecode "${UPD["result,${num},inline_query,from,last_name"]}")"
+	iQUERY[USERNAME]="$(JsonDecode "${UPD["result,${num},inline_query,from,username"]}")"
+	# always true
+	return 0
+}
+
+process_inline_button() {
+	local num="$1"
+	iBUTTON[DATA]="${UPD["result,${num},callback_query,data"]}"
+	iBUTTON[CHAT_ID]="${UPD["result,${num},callback_query,message,chat,id"]}"
+	iBUTTON[MESSAGE_ID]="${UPD["result,${num},callback_query,message,message_id"]}"
+	iBUTTON[MESSAGE]="$(JsonDecode "${UPD["result,${num},callback_query,message,text"]}")"
+# XXX should we give back pressed button, all buttons or nothing?
+	iBUTTON[USER_ID]="${UPD["result,${num},callback_query,from,id"]}"
+	iBUTTON[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},callback_query,from,first_name"]}")"
+	iBUTTON[LAST_NAME]="$(JsonDecode "${UPD["result,${num},callback_query,from,last_name"]}")"
+	iBUTTON[USERNAME]="$(JsonDecode "${UPD["result,${num},callback_query,from,username"]}")"
+	# always true
+	return 0
+}
+
+process_message() {
+	local num="$1"
+	# Message
+	MESSAGE[0]+="$(JsonDecode "${UPD["result,${num},message,text"]}" | sed 's|\\/|/|g')"
+	MESSAGE[ID]="${UPD["result,${num},message,message_id"]}"
+	MESSAGE[CAPTION]="$(JsonDecode "${UPD["result,${num},message,caption"]}")"
+	CAPTION="${MESSAGE[CAPTION]}"	# backward compatibility 
+	# dice received
+	MESSAGE[DICE]="${UPD["result,${num},message,dice,emoji"]}"
+	if [ -n "${MESSAGE[DICE]}" ]; then
+		MESSAGE[RESULT]="${UPD["result,${num},message,dice,value"]}"
+		MESSAGE[0]="/_dice_received ${MESSAGE[DICE]} ${MESSAGE[RESULT]}"
+	fi
+	# Chat ID is now parsed when update is received
+	CHAT[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,chat,last_name"]}")"
+	CHAT[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,chat,first_name"]}")"
+	CHAT[USERNAME]="$(JsonDecode "${UPD["result,${num},message,chat,username"]}")"
+	# set real name as username if empty
+	[ -z "${CHAT[USERNAME]}" ] && CHAT[USERNAME]="${CHAT[FIRST_NAME]} ${CHAT[LAST_NAME]}"
+	CHAT[TITLE]="$(JsonDecode "${UPD["result,${num},message,chat,title"]}")"
+	CHAT[TYPE]="$(JsonDecode "${UPD["result,${num},message,chat,type"]}")"
+	CHAT[ALL_ADMIN]="${UPD["result,${num},message,chat,all_members_are_administrators"]}"
+
+	# user ID is now parsed when update is received
+	USER[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,from,first_name"]}")"
+	USER[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,from,last_name"]}")"
+	USER[USERNAME]="$(JsonDecode "${UPD["result,${num},message,from,username"]}")"
+	# set real name as username if empty
+	[ -z "${USER[USERNAME]}" ] && USER[USERNAME]="${USER[FIRST_NAME]} ${USER[LAST_NAME]}"
+
+	# in reply to message from
+	if [ -n "${UPD["result,${num},message,reply_to_message,from,id"]}" ]; then
+	   REPLYTO[UID]="${UPD["result,${num},message,reply_to_message,from,id"]}"
+	   REPLYTO[0]="$(JsonDecode "${UPD["result,${num},message,reply_to_message,text"]}")"
+	   REPLYTO[ID]="${UPD["result,${num},message,reply_to_message,message_id"]}"
+	   REPLYTO[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,reply_to_message,from,first_name"]}")"
+	   REPLYTO[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,reply_to_message,from,last_name"]}")"
+	   REPLYTO[USERNAME]="$(JsonDecode "${UPD["result,${num},message,reply_to_message,from,username"]}")"
+	fi
+
+	# forwarded message from
+	if [ -n "${UPD["result,${num},message,forward_from,id"]}" ]; then
+	   FORWARD[UID]="${UPD["result,${num},message,forward_from,id"]}"
+	   FORWARD[ID]="${MESSAGE[ID]}"	# same as message ID
+	   FORWARD[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,forward_from,first_name"]}")"
+	   FORWARD[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,forward_from,last_name"]}")"
+	   FORWARD[USERNAME]="$(JsonDecode "${UPD["result,${num},message,forward_from,username"]}")"
+	fi
+
+	# get file URL from telegram, check for any of them!
+	if grep -qs -e '\["result",'"${num}"',"message","[avpsd].*,"file_id"\]' <<<"${UPDATE}"; then
+	    URLS[AUDIO]="$(get_file "${UPD["result,${num},message,audio,file_id"]}")"
+	    URLS[DOCUMENT]="$(get_file "${UPD["result,${num},message,document,file_id"]}")"
+	    URLS[PHOTO]="$(get_file "${UPD["result,${num},message,photo,0,file_id"]}")"
+	    URLS[STICKER]="$(get_file "${UPD["result,${num},message,sticker,file_id"]}")"
+	    URLS[VIDEO]="$(get_file "${UPD["result,${num},message,video,file_id"]}")"
+	    URLS[VOICE]="$(get_file "${UPD["result,${num},message,voice,file_id"]}")"
+	fi
+	# Contact, must have phone_number
+	if [ -n "${UPD["result,${num},message,contact,phone_number"]}" ]; then
+		CONTACT[USER_ID]="$(JsonDecode  "${UPD["result,${num},message,contact,user_id"]}")"
+		CONTACT[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,contact,first_name"]}")"
+		CONTACT[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,contact,last_name"]}")"
+		CONTACT[NUMBER]="${UPD["result,${num},message,contact,phone_number"]}"
+		CONTACT[VCARD]="${UPD["result,${num},message,contact,vcard"]}"
+	fi
+
+	# venue, must have a position
+	if [ -n "${UPD["result,${num},message,venue,location,longitude"]}" ]; then
+		VENUE[TITLE]="$(JsonDecode "${UPD["result,${num},message,venue,title"]}")"
+		VENUE[ADDRESS]="$(JsonDecode "${UPD["result,${num},message,venue,address"]}")"
+		VENUE[LONGITUDE]="${UPD["result,${num},message,venue,location,longitude"]}"
+		VENUE[LATITUDE]="${UPD["result,${num},message,venue,location,latitude"]}"
+		VENUE[FOURSQUARE]="${UPD["result,${num},message,venue,foursquare_id"]}"
+	fi
+
+	# Location
+	LOCATION[LONGITUDE]="${UPD["result,${num},message,location,longitude"]}"
+	LOCATION[LATITUDE]="${UPD["result,${num},message,location,latitude"]}"
+
+	# service messages, group or channel only!
+	if [[ "${CHAT[ID]}" == "-"* ]] ; then
+	    # new chat member
+	    if [ -n "${UPD["result,${num},message,new_chat_member,id"]}" ]; then
+		SERVICE[NEWMEMBER]="${UPD["result,${num},message,new_chat_member,id"]}"
+		NEWMEMBER[ID]="${SERVICE[NEWMEMBER]}"
+		NEWMEMBER[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,new_chat_member,first_name"]}")"
+		NEWMEMBER[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,new_chat_member,last_name"]}")"
+		NEWMEMBER[USERNAME]="$(JsonDecode "${UPD["result,${num},message,new_chat_member,username"]}")"
+		NEWMEMBER[ISBOT]="${UPD["result,${num},message,new_chat_member,is_bot"]}"
+		MESSAGE[0]="/_new_chat_member ${NEWMEMBER[ID]} ${NEWMEMBER[USERNAME]:=${NEWMEMBER[FIRST_NAME]} ${NEWMEMBER[LAST_NAME]}}"
+	    fi
+	    # left chat member
+	    if [ -n "${UPD["result,${num},message,left_chat_member,id"]}" ]; then
+		SERVICE[LEFTMEMBER]="${UPD["result,${num},message,left_chat_member,id"]}"
+		LEFTMEMBER[ID]="${SERVICE[LEFTMEBER]}"
+		LEFTMEMBER[FIRST_NAME]="$(JsonDecode "${UPD["result,${num},message,left_chat_member,first_name"]}")"
+		LEFTMEMBER[LAST_NAME]="$(JsonDecode "${UPD["result,${num},message,left_chat_member,last_name"]}")"
+		LEFTMEBER[USERNAME]="$(JsonDecode "${UPD["result,${num},message,left_chat_member,username"]}")"
+		LEFTMEMBER[ISBOT]="${UPD["result,${num},message,left_chat_member,is_bot"]}"
+		MESSAGE[0]="/_left_chat_member ${LEFTMEMBER[ID]} ${LEFTMEMBER[USERNAME]:=${LEFTMEMBER[FIRST_NAME]} ${LEFTMEMBER[LAST_NAME]}}"
+	    fi
+	    # chat title / photo, check for any of them!
+	    if grep -qs -e '\["result",'"${num}"',"message","new_chat_[tp]' <<<"${UPDATE}"; then
+		SERVICE[NEWTITLE]="$(JsonDecode "${UPD["result,${num},message,new_chat_title"]}")"
+		[ -n "${SERVICE[NEWTITLE]}" ] &&\
+			MESSAGE[0]="/_new_chat_title ${USER[ID]} ${SERVICE[NEWTITLE]}"
+		SERVICE[NEWPHOTO]="$(get_file "${UPD["result,${num},message,new_chat_photo,0,file_id"]}")"
+		[ -n "${SERVICE[NEWPHOTO]}" ] &&\
+			 MESSAGE[0]="/_new_chat_photo ${USER[ID]} ${SERVICE[NEWPHOTO]}"
+	    fi
+	    # pinned message
+	    if [ -n "${UPD["result,${num},message,pinned_message,message_id"]}" ]; then
+		SERVICE[PINNED]="${UPD["result,${num},message,pinned_message,message_id"]}"
+		PINNED[ID]="${SERVICE[PINNED]}"
+		PINNED[MESSAGE]="$(JsonDecode "${UPD["result,${num},message,pinned_message,text"]}")"
+		MESSAGE[0]="/_new_pinned_message ${USER[ID]} ${PINNED[ID]} ${PINNED[MESSAGE]}"
+	    fi
+	    # migrate to super group
+	    if [ -n "${UPD["result,${num},message,migrate_to_chat_id"]}" ]; then
+		MIGRATE[TO]="${UPD["result,${num},message,migrate_to_chat_id"]}"
+		MIGRATE[FROM]="${UPD["result,${num},message,migrate_from_chat_id"]}"
+		# CHAT is already migrated, so set new chat id
+		[ "${CHAT[ID]}" = "${MIGRATE[FROM]}" ] && CHAT[ID]="${MIGRATE[FROM]}"
+		SERVICE[MIGRATE]="${MIGRATE[FROM]} ${MIGRATE[TO]}"
+		MESSAGE[0]="/_migrate_group ${SERVICE[MIGRATE]}"
+	    fi
+	    # set SERVICE to yes if a service message was received
+	    [[ "${SERVICE[*]}" =~  ^[[:blank:]]*$ ]] || SERVICE[0]="yes"
+	fi
+
+	# split message in command and args
+	[[ "${MESSAGE[0]}" == "/"* ]] && read -ra CMD <<<"${MESSAGE[0]}" &&  CMD[0]="${CMD[0]%%@*}"
+	# everything went well
+	return 0
+}
+
+#########################
+# bot startup actions, call before start polling or webhook loop
+declare -A BASHBOTBLOCKED
+start_bot() {
+	local DEBUGMSG
+	# startup message
+	DEBUGMSG="BASHBOT startup actions, mode set to \"${1:-normal}\" =========="
+	log_update "${DEBUGMSG}"
+	# redirect to Debug.log
+	if [[ "$1" == *"debug" ]]; then
+		# shellcheck disable=SC2153
+		exec &>>"${DEBUGLOG}"
+		log_debug "${DEBUGMSG}";
+	fi
+	DEBUGMSG="$1"
+	[[ "${DEBUGMSG}" == "xdebug"* ]] && set -x
+	# cleaup old pipes and empty logfiles
+	find "${DATADIR}" -type p -not -name "webhook-fifo-*" -delete
+	find "${DATADIR}" -size 0 -name "*.log" -delete
+	# load addons on startup
+	for addons in "${ADDONDIR:-.}"/*.sh ; do
+		# shellcheck disable=SC1090
+		[ -r "${addons}" ] && source "${addons}" "startbot" "${DEBUGMSG}"
+	done
+	# shellcheck disable=SC1090
+	source "${COMMANDS}" "startbot"
+	# start timer events
+	if [ -n "${BASHBOT_START_TIMER}" ] ; then
+		# shellcheck disable=SC2064
+		trap "event_timer ${DEBUGMSG}" ALRM
+		start_timer &
+		# shellcheck disable=SC2064
+		trap "kill -9 $!; exit" EXIT INT HUP TERM QUIT 
+	fi
+	# cleanup on start
+	bot_cleanup "startup"
+	# read blocked users
+	jssh_readDB_async "BASHBOTBLOCKED" "${BLOCKEDFILE}"
+	# inform botadmin about start
+	send_normal_message "$(getConfigKey "botadmin")" "Bot ${ME} $2 started ..." &
+}
+
+# main polling updates loop, should never terminate
+get_updates(){
+	local errsleep="200" DEBUG="$1" OFFSET=0
+	# adaptive sleep defaults
+	local nextsleep="100"
+	local stepsleep="${BASHBOT_SLEEP_STEP:-100}"
+	local maxsleep="${BASHBOT_SLEEP:-5000}"
+	printf "%(%c)T: %b\n" -1 "Bot startup actions done, start polling updates ..."
+	while true; do
+		# adaptive sleep in ms rounded to next 0.1 s
+		sleep "$(_round_float "${nextsleep}e-3" "1")"
+		# get next update
+		# shellcheck disable=SC2153
+		UPDATE="$(getJson "${URL}/getUpdates?offset=${OFFSET}" 2>/dev/null | "${JSONSHFILE}" -b -n 2>/dev/null | iconv -f utf-8 -t utf-8 -c)"
+		# did we get an response?
+		if [ -n "${UPDATE}" ]; then
+			# we got something, do processing
+			[ "${OFFSET}" = "-999" ] && [ "${nextsleep}" -gt "$((maxsleep*2))" ] &&\
+				log_error "Recovered from timeout/broken/no connection, continue with telegram updates"
+			# calculate next sleep interval
+			((nextsleep+= stepsleep , nextsleep= nextsleep>maxsleep ?maxsleep:nextsleep))
+			# warn if webhook is set
+			if grep -q '^\["error_code"\]	409' <<<"${UPDATE}"; then
+				[ "${OFFSET}" != "-999" ] && nextsleep="${stepsleep}"
+				OFFSET="-999"; errsleep="$(_round_float "$(( errsleep= 300*nextsleep ))e-3")"
+				log_error "Warning conflicting webhook set, can't get updates until your run delete_webhook! Sleep $((errsleep/60)) min ..."
+				sleep "${errsleep}"
+				continue
+			fi
+			# Offset
+			OFFSET="$(grep <<<"${UPDATE}" '\["result",[0-9]*,"update_id"\]' | tail -1 | cut -f 2)"
+			((OFFSET++))
+
+			if [ "${OFFSET}" != "1" ]; then
+				nextsleep="100"
+				process_multi_updates "${DEBUG}"
+			fi
+		else
+			# oops, something bad happened, wait maxsleep*10
+			(( nextsleep=nextsleep*2 , nextsleep= nextsleep>maxsleep*10 ?maxsleep*10:nextsleep ))
+			# second time, report problem
+			if [ "${OFFSET}" = "-999" ]; then
+			    log_error "Repeated timeout/broken/no connection on telegram update, sleep $(_round_float "${nextsleep}e-3")s"
+			    # try to recover
+			    if _is_function bashbotBlockRecover && [ -z "$(getJson "${ME_URL}")" ]; then
+				log_error "Try to recover, calling bashbotBlockRecover ..."
+				bashbotBlockRecover >>"${ERRORLOG}"
+			    fi
+			fi
+			OFFSET="-999"
+		fi
+	done
+}
+
+
+
+declare -Ax BASHBOT_EVENT_INLINE BASHBOT_EVENT_MESSAGE BASHBOT_EVENT_CMD BASHBOT_EVENT_REPLYTO BASHBOT_EVENT_FORWARD BASHBOT_EVENT_SEND
+declare -Ax BASHBOT_EVENT_CONTACT BASHBOT_EVENT_LOCATION BASHBOT_EVENT_FILE BASHBOT_EVENT_TEXT BASHBOT_EVENT_TIMER BASHBOT_BLOCKED
+
+start_timer(){
+	# send alarm every ~60 s
+	while :; do
+		sleep 59.5
+    		kill -ALRM $$
+	done;
+}
+
+EVENT_TIMER="0"
+event_timer() {
+	local key timer debug="$1"
+	(( EVENT_TIMER++ ))
+	# shellcheck disable=SC2153
+	for key in "${!BASHBOT_EVENT_TIMER[@]}"
+	do
+		timer="${key##*,}"
+		[[ ! "${timer}" =~ ^-*[1-9][0-9]*$ ]] && continue
+		if [ "$(( EVENT_TIMER % timer ))" = "0" ]; then
+			_exec_if_function "${BASHBOT_EVENT_TIMER[${key}]}" "timer" "${key}" "${debug}"
+			[ "$(( EVENT_TIMER % timer ))" -lt "0" ] && \
+				unset BASHBOT_EVENT_TIMER["${key}"]
+		fi
+	done
+}
+
+event_inline() {
+	local key debug="$1"
+	# shellcheck disable=SC2153
+	for key in "${!BASHBOT_EVENT_INLINE[@]}"
+	do
+		_exec_if_function "${BASHBOT_EVENT_INLINE[${key}]}" "inline" "${key}" "${debug}"
+	done
+}
+event_message() {
+	local key debug="$1"
+	# ${MESSAEG[*]} event_message
+	# shellcheck disable=SC2153
+	for key in "${!BASHBOT_EVENT_MESSAGE[@]}"
+	do
+		 _exec_if_function "${BASHBOT_EVENT_MESSAGE[${key}]}" "message" "${key}" "${debug}"
+	done
+	
+	# ${TEXT[*]} event_text
+	if [ -n "${MESSAGE[0]}" ]; then
+		# shellcheck disable=SC2153
+		for key in "${!BASHBOT_EVENT_TEXT[@]}"
+		do
+			_exec_if_function "${BASHBOT_EVENT_TEXT[${key}]}" "text" "${key}" "${debug}"
+		done
+
+		# ${CMD[*]} event_cmd
+		if [ -n "${CMD[0]}" ]; then
+			# shellcheck disable=SC2153
+			for key in "${!BASHBOT_EVENT_CMD[@]}"
+			do
+				_exec_if_function "${BASHBOT_EVENT_CMD[${key}]}" "command" "${key}" "${debug}"
+			done
+		fi
+	fi
+	# ${REPLYTO[*]} event_replyto
+	if [ -n "${REPLYTO[UID]}" ]; then
+		# shellcheck disable=SC2153
+		for key in "${!BASHBOT_EVENT_REPLYTO[@]}"
+		do
+			_exec_if_function "${BASHBOT_EVENT_REPLYTO[${key}]}" "replyto" "${key}" "${debug}"
+		done
+	fi
+
+	# ${FORWARD[*]} event_forward
+	if [ -n "${FORWARD[UID]}" ]; then
+		# shellcheck disable=SC2153
+		for key in "${!BASHBOT_EVENT_FORWARD[@]}"
+		do
+			 _exec_if_function && "${BASHBOT_EVENT_FORWARD[${key}]}" "forward" "${key}" "${debug}"
+		done
+	fi
+
+	# ${CONTACT[*]} event_contact
+	if [ -n "${CONTACT[FIRST_NAME]}" ]; then
+		# shellcheck disable=SC2153
+		for key in "${!BASHBOT_EVENT_CONTACT[@]}"
+		do
+			_exec_if_function "${BASHBOT_EVENT_CONTACT[${key}]}" "contact" "${key}" "${debug}"
+		done
+	fi
+
+	# ${VENUE[*]} event_location
+	# ${LOCATION[*]} event_location
+	if [ -n "${LOCATION[LONGITUDE]}" ] || [ -n "${VENUE[TITLE]}" ]; then
+		# shellcheck disable=SC2153
+		for key in "${!BASHBOT_EVENT_LOCATION[@]}"
+		do
+			_exec_if_function "${BASHBOT_EVENT_LOCATION[${key}]}" "location" "${key}" "${debug}"
+		done
+	fi
+
+	# ${URLS[*]} event_file
+	# NOTE: compare again #URLS -1 blanks!
+	if [[ "${URLS[*]}" != "     " ]]; then
+		# shellcheck disable=SC2153
+		for key in "${!BASHBOT_EVENT_FILE[@]}"
+		do
+			_exec_if_function "${BASHBOT_EVENT_FILE[${key}]}" "file" "${key}" "${debug}"
+		done
+	fi
+
+}
+

modules/sendMessage.sh

@@ -5,7 +5,8 @@
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+# shellcheck disable=SC1117
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # will be automatically sourced from bashbot
 
@@ -14,294 +15,448 @@ eval "$(basename "${BASH_SOURCE[0]}")(){ :; }"
 
 # source from commands.sh to use the sendMessage functions
 
-MSG_URL=$URL'/sendMessage'
-PHO_URL=$URL'/sendPhoto'
-AUDIO_URL=$URL'/sendAudio'
-DOCUMENT_URL=$URL'/sendDocument'
-STICKER_URL=$URL'/sendSticker'
-VIDEO_URL=$URL'/sendVideo'
-VOICE_URL=$URL'/sendVoice'
-LOCATION_URL=$URL'/sendLocation'
-VENUE_URL=$URL'/sendVenue'
-ACTION_URL=$URL'/sendChatAction'
-FORWARD_URL=$URL'/forwardMessage'
-ALBUM_URL=$URL'/sendMediaGroup'
+MSG_URL=${URL}'/sendMessage'
+EDIT_URL=${URL}'/editMessageText'
+
+#
+# send/edit message variants ------------------
+#
 
 # $1 CHAT $2 message
 send_normal_message() {
-	local len text; text="$(JsonEscape "${2}")"
-	text="${text//$'\n'/\\n}"
+	local len text; text="$(JsonEscape "$2")"
 	until [ -z "${text}" ]; do
 		if [ "${#text}" -le 4096 ]; then
-			sendJson "${1}" '"text":"'"${text}"'"' "${MSG_URL}"
+			sendJson "$1" '"text":"'"${text}"'"' "${MSG_URL}"
 			break
 		else
 			len=4095
 			[ "${text:4095:2}" != "\n" ] &&\
 				len="${text:0:4096}" && len="${len%\\n*}" && len="${#len}"
-			sendJson "${1}" '"text":"'"${text:0:${len}}"'"' "${MSG_URL}"
+			sendJson "$1" '"text":"'"${text:0:${len}}"'"' "${MSG_URL}"
 			text="${text:$((len+2))}"
 		fi
 	done
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
 }
 
 # $1 CHAT $2 message
 send_markdown_message() {
-	local text; text="$(JsonEscape "${2}")"
-	text="${text//$'\n'/\\n}"
-	[ "${#text}" -ge 4096 ] && log_error "Warning: markdown message longer than 4096 characters, message is rejected if formatting crosses 4096 border."
-	until [ -z "${text}" ]; do
-		sendJson "${1}" '"text":"'"${text:0:4096}"'","parse_mode":"markdown"' "${MSG_URL}"
-		text="${text:4096}"
-	done
+	_format_message_url "$1" "$2" ',"parse_mode":"markdown"' "${MSG_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
 }
 
 # $1 CHAT $2 message
 send_markdownv2_message() {
-	local text; text="$(JsonEscape "${2}")"
-	text="${text//$'\n'/\\n}"
-	[ "${#text}" -ge 4096 ] && log_error "Warning: markdown message longer than 4096 characters, message is rejected if formatting crosses 4096 border."
-	# markdown v2 needs additional double escaping!
-	text="$(sed -E -e 's|([#{}()!.-])|\\\1|g' <<< "$text")"
-	until [ -z "${text}" ]; do
-		sendJson "${1}" '"text":"'"${text:0:4096}"'","parse_mode":"markdownv2"' "${MSG_URL}"
-		text="${text:4096}"
-	done
+	_markdownv2_message_url "$1" "$2" ',"parse_mode":"markdownv2"' "${MSG_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
 }
 
 # $1 CHAT $2 message
 send_html_message() {
-	local text; text="$(JsonEscape "${2}")"
-	text="${text//$'\n'/\\n}"
-	[ "${#text}" -ge 4096 ] && log_error "Warning: html message longer than 4096 characters, message is rejected if formatting crosses 4096 border."
+	_format_message_url "$1" "$2" ',"parse_mode":"html"' "${MSG_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
+}
+
+# $1 CHAT $2 msg-id $3 message
+edit_normal_message() {
+	_format_message_url "$1" "$3" ',"message_id":'"$2"'' "${EDIT_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
+}
+
+# $1 CHAT $2 msg-id $3 message
+edit_markdown_message() {
+	_format_message_url "$1" "$3" ',"message_id":'"$2"',"parse_mode":"markdown"' "${EDIT_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
+}
+
+# $1 CHAT $2 msg-id $3 message
+edit_markdownv2_message() {
+	_markdownv2_message_url "$1" "$3" ',"message_id":'"$2"',"parse_mode":"markdownv2"' "${EDIT_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
+}
+
+# $1 CHAT $2 msg-id $3 message
+edit_html_message() {
+	_format_message_url "$1" "$3" ',"message_id":'"$2"',"parse_mode":"html"' "${EDIT_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
+}
+
+# $1 chat $2 mesage_id, $3 caption
+edit_message_caption() {
+	sendJson "$1" '"message_id":'"$2"',"caption":"'"$3"'"' "${URL}/editMessageCaption"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
+}
+
+
+# $ chat $2 msg_id $3 nolog
+delete_message() {
+	[ -z "$3" ] && log_update "Delete Message CHAT=$1 MSG_ID=$2"
+	sendJson "$1" '"message_id": '"$2"'' "${URL}/deleteMessage"
+	[ "${BOTSENT[OK]}" = "true" ] && BOTSENT[CHAT]="$1"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
+}
+
+
+# internal function, send/edit formatted message with parse_mode and URL
+# $1 CHAT $2 message $3 action $4 URL
+_format_message_url(){
+	local text; text="$(JsonEscape "$2")"
+	[ "${#text}" -ge 4096 ] && log_error "Warning: html/markdown message longer than 4096 characters, message is rejected if formatting crosses 4096 border."
 	until [ -z "${text}" ]; do
-		sendJson "${1}" '"text":"'"${text:0:4096}"'","parse_mode":"html"' "${MSG_URL}"
+		sendJson "$1" '"text":"'"${text:0:4096}"'"'"$3"'' "$4"
 		text="${text:4096}"
 	done
 }
 
-# obsolote, will be removed after 1.0!!
-# $1 CHAT $2 message $3 keyboard
-old_send_keyboard() {
-	local text; text='"text":"'$(JsonEscape "${2}")'"'
-	shift 2
-	local keyboard="init"
-	OLDIFS="$IFS"
-	IFS="\""
-	for f in "$@" ;do [ "$f" != " " ] && keyboard="$keyboard, [\"$f\"]";done
-	IFS="$OLDIFS"
-	keyboard="${keyboard/init, /}"
-	sendJson "${1}" "${text}"', "reply_markup": {"keyboard": [ '"${keyboard}"' ],"one_time_keyboard": true}' "$MSG_URL"
-	send_normal_message "$(getConfigKey "botadmin")" "Warning: old 'send_keyboard' format is deprecated since version 0.6 and will be removed after 1.0 release!"
+# internal function, send/edit markdownv2 message with URL
+# $1 CHAT $2 message $3 action $4 URL
+_markdownv2_message_url() {
+	local text; text="$(JsonEscape "$2")"
+	[ "${#text}" -ge 4096 ] && log_error "Warning: markdownv2 message longer than 4096 characters, message is rejected if formatting crosses 4096 border."
+	# markdown v2 needs additional double escaping!
+	text="$(sed -E -e 's|([_|~`>+=#{}()!.-])|\\\1|g' <<< "${text}")"
+	until [ -z "${text}" ]; do
+		sendJson "$1" '"text":"'"${text:0:4096}"'"'"$3"'' "$4"
+		text="${text:4096}"
+	done
 }
 
+#
+# send keyboard, buttons, files ---------------
+#
+
 # $1 CHAT $2 message $3 keyboard
 send_keyboard() {
 	if [[ "$3" != *'['* ]]; then old_send_keyboard "${@}"; return; fi
 	local text='"text":"'"Keyboard:"'"'
-	if [ -n "${2}" ]; then
-		text="$(JsonEscape "${2}")"
+	if [ -n "$2" ]; then
+		text="$(JsonEscape "$2")"
 		text='"text":"'"${text//$'\n'/\\n}"'"'
 	fi
+	# text longer than 4096, send text and keyboard separate
+	if [ "${#2}" -gt 4096 ]; then
+		send_normal_message "$1" "$2"
+		send_keyboard "$1" "..." "$3"
+		return
+	fi
 	local one_time=', "one_time_keyboard":true' && [ -n "$4" ] && one_time=""
-	sendJson "${1}" "${text}"', "reply_markup": {"keyboard": [ '"${3}"' ] '"${one_time}"'}' "$MSG_URL"
-	# '"text":"$2", "reply_markup": {"keyboard": [ ${3} ], "one_time_keyboard": true}'
+	# '"text":"$2", "reply_markup": {"keyboard": [ $3 ], "one_time_keyboard": true}'
+	sendJson "$1" "${text}"', "reply_markup": {"keyboard": [ '"$3"' ] '"${one_time}"'}' "${MSG_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
 }
 
 # $1 CHAT $2 message $3 remove
 remove_keyboard() {
 	local text='"text":"'"remove custom keyboard ..."'"'
-	if [ -n "${2}" ]; then
-		text="$(JsonEscape "${2}")"
+	if [ -n "$2" ]; then
+		text="$(JsonEscape "$2")"
 		text='"text":"'"${text//$'\n'/\\n}"'"'
 	fi
-	sendJson "${1}" "${text}"', "reply_markup": {"remove_keyboard":true}' "$MSG_URL"
+	sendJson "$1" "${text}"', "reply_markup": {"remove_keyboard":true}' "${MSG_URL}"
 	# delete message if no message or $3 not empty
-	[[ -z "${2}" || -n "${3}" ]] && delete_message "${1}" "${BOTSENT[ID]}" "nolog"
 	#JSON='"text":"$2", "reply_markup": {"remove_keyboard":true}'
+	[[ -z "$2" || -n "$3" ]] && delete_message "$1" "${BOTSENT[ID]}" "nolog"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
 }
 
+# buttons will specified as "texts
+#|url" ... "text|url" empty arg starts new row
+# url not starting with http:// or https:// will be send as callback_data 
+send_inline_buttons(){
+	send_inline_keyboard "$1" "$2" "$(_button_row "${@:3}")"
+}
+
+# $1 CHAT $2 message-id $3 buttons
+# buttons will specified as "text|url" ... "text|url" empty arg starts new row
+# url not starting with http:// or https:// will be send as callback_data 
+edit_inline_buttons(){
+	edit_inline_keyboard "$1" "$2" "$(_button_row "${@:3}")"
+}
+
+
+# $1 CHAT $2 message $3 button text $4 button url
+send_button() {
+	send_inline_keyboard "$1" "$2" '[{"text":"'"$(JsonEscape "$3")"'", "url":"'"$4"'"}]'
+}
+
+# helper function to create json for a button row
+# buttons will specified as "text|url" ... "text|url" empty arg starts new row
+# url not starting with http:// or https:// will be send as callback_data 
+_button_row() {
+	[ -z "$1" ] && return 1
+	local arg type json sep
+	for arg in "$@"
+	do
+		[ -z "${arg}" ] && sep="],[" && continue
+		type="callback_data"
+		[[ "${arg##*|}" =~ ^(https*://|tg://) ]] && type="url"
+		json+="${sep}"'{"text":"'"$(JsonEscape "${arg%|*}")"'", "'"${type}"'":"'"${arg##*|}"'"}'
+		sep=","
+	done
+	printf "[%s]" "${json}"
+}
+
+# raw inline functions, for special use
+# $1 CHAT $2 message-id $3 keyboard
+edit_inline_keyboard() {
+	# JSON='"message_id":"$2", "reply_markup": {"inline_keyboard": [ $3->[{"text":"text", "url":"url"}]<- ]}'
+	sendJson "$1" '"message_id":'"$2"', "reply_markup": {"inline_keyboard": [ '"$3"' ]}' "${URL}/editMessageReplyMarkup"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
+}
+
+
 # $1 CHAT $2 message $3 keyboard
 send_inline_keyboard() {
-	local text; text='"text":"'$(JsonEscape "${2}")'"'; [ -z "${2}" ] && text='"text":"'"Keyboard:"'"'
-	sendJson "${1}" "${text}"', "reply_markup": {"inline_keyboard": [ '"${3}"' ]}' "$MSG_URL"
-	# JSON='"text":"$2", "reply_markup": {"inline_keyboard": [ $3->[{"text":"text", "url":"url"}]<- ]}'
+	local text; text='"text":"'$(JsonEscape "$2")'"'; [ -z "$2" ] && text='"text":"..."'
+	sendJson "$1" "${text}"', "reply_markup": {"inline_keyboard": [ '"$3"' ]}' "${MSG_URL}"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
 }
-# $1 CHAT $2 message $3 button text $4 URL
-send_button() {
-	send_inline_keyboard "${1}" "${2}" '[ {"text":"'"$(JsonEscape "${3}")"'", "url":"'"${4}"'"}]' 
+
+# $1 callback id, $2 text to show, alert if not empty
+answer_callback_query() {
+	local alert
+	[ -n "$3" ] && alert='","show_alert": true'
+	sendJson "" '"callback_query_id": "'"$1"'","text":"'"$2${alert}"'"' "${URL}/answerCallbackQuery"
+}
+
+# $1 chat, $2 file_id on telegram server 
+send_sticker() {
+	sendJson "$1" '"sticker": "'"$2"'"' "${URL}/sendSticker"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
 }
 
 
-UPLOADDIR="${BASHBOT_UPLOAD:-${DATADIR}/upload}"
-
-# for now this can only send local files with curl!
-# extend to allow send files by URL or telegram ID
-send_file() {
-	upload_file "${@}"
-}
-
-if [ -z "${BASHBOT_WGET}" ] && _exists curl ; then
-# there are no checks if URL or ID exists
-# $1 chat $3 ... $n URL or ID
+# only curl can send files ... 
+if detect_curl ; then
+  # there are no checks if URL or ID exists
+  # $1 chat $3 ... $n URL or ID
   send_album(){
-	[ -z "${1}" ] && return 1
-	[ -z "${3}" ] && return 2 # minimum 2 files
-	local CHAT JSON IMAGE; CHAT="${1}"; shift 
+	[ -z "$1" ] && return 1
+	[ -z "$3" ] && return 2	# minimum 2 files
+	local CHAT JSON IMAGE; CHAT="$1"; shift 
 	for IMAGE in "$@"
 	do
 		[ -n "${JSON}" ] && JSON+=","
 		JSON+='{"type":"photo","media":"'${IMAGE}'"}'
 	done
 	# shellcheck disable=SC2086
-	res="$("${BASHBOT_CURL}" -s -k ${BASHBOT_CURL_ARGS} "${ALBUM_URL}" -F "chat_id=${CHAT}"\
+	res="$("${BASHBOT_CURL}" -s -k ${BASHBOT_CURL_ARGS} "${URL}/sendMediaGroup" -F "chat_id=${CHAT}"\
 			-F "media=[${JSON}]" | "${JSONSHFILE}" -s -b -n 2>/dev/null )"
 	sendJsonResult "${res}" "send_album (curl)" "${CHAT}" "$@"
 	[[ -z "${SOURCE}" && -n "${BASHBOT_EVENT_SEND[*]}" ]] && event_send "album" "$@" &
   }
 else
   send_album(){
-	log_error "Sorry, wget Album upload not yet implemented"
+	log_error "Sorry, wget Album upload not implemented"
 	BOTSENT[OK]="false"
 	[[ -z "${SOURCE}" && -n "${BASHBOT_EVENT_SEND[*]}" ]] && event_send "album" "$@" &
   }
 fi
 
-upload_file(){
-	local CUR_URL WHAT STATUS file="$2"
-	# file access checks ...
-	[[ "$file" = *'..'* ]] && return  # no directory traversal
-	[[ "$file" = '.'* ]] && return	 # no hidden or relative files
-	if [[ "$file" = '/'* ]] ; then
-		[[ ! "$file" =~ $FILE_REGEX ]] && return # absolute must match REGEX
+# supports local file, URL and file_id
+# $1 chat, $2 file https::// file_id:// , $3 caption, $4 extension (optional)
+send_file(){
+	local url what num stat media capt="$3" file="$2" ext="$4"
+	if [[ "${file}" =~ ^https*:// ]]; then
+		media="URL"
+	elif [[ "${file}" == file_id://* ]]; then
+		media="ID"
+		file="${file#file_id://}"
 	else
-		file="${UPLOADDIR:-NOUPLOADDIR}/${file}" # othiers must be in UPLOADDIR
+		# we have a file, check file location ...
+		media="FILE"
+		file="$(checkUploadFile "$1" "$2" "send_file")"
+		[ -z "${file}" ] && return 1
+		# file OK, let's continue
 	fi
-	[ ! -r "$file" ] && return # and file must exits of course
- 
-	local ext="${file##*.}"
-	case $ext in
-        	mp3|flac)
-			CUR_URL="$AUDIO_URL"
-			WHAT="audio"
-			STATUS="upload_audio"
-			;;
-		png|jpg|jpeg|gif|pic)
-			CUR_URL="$PHO_URL"
-			WHAT="photo"
-			STATUS="upload_photo"
-			;;
-		webp)
-			CUR_URL="$STICKER_URL"
-			WHAT="sticker"
-			STATUS="upload_photo"
-			;;
-		mp4)
-			CUR_URL="$VIDEO_URL"
-			WHAT="video"
-			STATUS="upload_video"
-			;;
 
-		ogg)
-			CUR_URL="$VOICE_URL"
-			WHAT="voice"
-			STATUS="upload_audio"
+	# no type given, use file ext, if no ext type photo
+	if [ -z "${ext}" ]; then
+		ext="${file##*.}"
+		[ "${ext}" = "${file}" ] && ext="photo"
+	fi
+	# select upload URL
+	case "${ext}" in
+		photo|png|jpg|jpeg|gif|pic)
+			url="${URL}/sendPhoto"; what="photo"; num=",0"; stat="upload_photo"
 			;;
-		*)
-			CUR_URL="$DOCUMENT_URL"
-			WHAT="document"
-			STATUS="upload_document"
+        	audio|mp3|flac)
+			url="${URL}/sendAudio"; what="audio"; stat="upload_audio"
+			;;
+		sticker|webp)
+			url="${URL}/sendSticker"; what="sticker"; stat="upload_photo"
+			;;
+		video|mp4)
+			url="${URL}/sendVideo"; what="video"; stat="upload_video"
+			;;
+		voice|ogg)
+			url="${URL}/sendVoice"; what="voice"; stat="record_audio"
+			;;
+		*)	url="${URL}/sendDocument"; what="document"; stat="upload_document"
 			;;
 	esac
-	send_action "${1}" "$STATUS"
-	sendUpload "$1" "${WHAT}" "${file}" "${CUR_URL}" "$3"
+
+	# show file upload to user
+	send_action "$1" "${stat}"
+	# select method to send
+	case "${media}" in
+		FILE)	# send local file ...
+			sendUpload "$1" "${what}" "${file}" "${url}" "${capt//\\n/$'\n'}";;
+
+		URL|ID)	# send URL, file_id ...
+			sendJson "$1" '"'"${what}"'":"'"${file}"'","caption":"'"${capt//\\n/$'\n'}"'"' "${url}"
+	esac
+	# get file_id and file_type
+	if [ "${BOTSENT[OK]}" = "true" ]; then
+		BOTSENT[FILE_ID]="${UPD["result,${what}${num},file_id"]}"
+		BOTSENT[FILE_TYPE]="${what}"
+	fi
+	return 0
 }
 
-# typing for text messages, upload_photo for photos, record_video or upload_video for videos, record_audio or upload_audio for audio files, upload_document for general files, find_location for location
+# $1 chat $2 typing upload_photo record_video upload_video record_audio upload_audio upload_document find_location
 send_action() {
 	[ -z "$2" ] && return
-	sendJson "${1}" '"action": "'"${2}"'"' "$ACTION_URL" &
+	sendJson "$1" '"action": "'"$2"'"' "${URL}/sendChatAction" &
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
+}
+
+# $1 chat $2 emoji “🎲”, “🎯”, “🏀”, “⚽”, “🎰" "🎳"
+# code: "\ud83c\udfb2" "\ud83c\udfaf" "\ud83c\udfc0" "\u26bd" "\ud83c\udfb0"
+# text: ":game_die:" ":dart:" ":basketball:" ":soccer:" :slot_machine:"
+# $3 reply_to_id
+send_dice() {
+	local reply emoji='\ud83c\udfb2'	# default "🎲"
+	[[ "$3" =~ ^[${o9o9o9}-]+$ ]] && reply=',"reply_to_message_id":'"$3"',"allow_sending_without_reply": true'
+	case "$2" in # convert input to single character emoji
+		*🎲*|*game*|*dice*|*'dfb2'*|*'DFB2'*)	: ;;
+		*🎯*|*dart*  |*'dfaf'*|*'DFAF'*)	emoji='\ud83c\udfaf' ;;
+		*🏀*|*basket*|*'dfc0'*|*'DFC0'*)	emoji='\ud83c\udfc0' ;;
+		*⚽*|*soccer*|*'26bd'*|*'26BD'*)	emoji='\u26bd' ;;
+		*🎰*|*slot*  |*'dfb0'*|*'DFB0'*)	emoji='\ud83c\udfb0' ;;
+		*🎳*|*bowl*  |*'dfb3'*|*'DFB3'*)	emoji='\ud83c\udfb3' ;;
+	esac
+	sendJson "$1" '"emoji": "'"${emoji}"'"'"${reply}" "${URL}/sendDice"
+	if [ "${BOTSENT[OK]}" = "true" ]; then
+		BOTSENT[DICE]="${UPD["result,dice,emoji"]}"
+		BOTSENT[RESULT]="${UPD["result,dice,value"]}"
+	else
+		# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+		processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2"
+	fi
 }
 
 # $1 CHAT $2 lat $3 long
 send_location() {
 	[ -z "$3" ] && return
-	sendJson "${1}" '"latitude": '"${2}"', "longitude": '"${3}"'' "$LOCATION_URL"
+	sendJson "$1" '"latitude": '"$2"', "longitude": '"$3"'' "${URL}/sendLocation"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
 }
 
-# $1 CHAT $2 lat $3 long $4 title $5 address $6 foursquard id
+# $1 CHAT $2 lat $3 long $4 title $5 address $6 foursquare id
 send_venue() {
 	local add=""
 	[ -z "$5" ] && return
 	[ -n "$6" ] && add=', "foursquare_id": '"$6"''
-	sendJson "${1}" '"latitude": '"${2}"', "longitude": '"${3}"', "address": "'"${5}"'", "title": "'"${4}"'"'"${add}" "$VENUE_URL"
+	sendJson "$1" '"latitude": '"$2"', "longitude": '"$3"', "address": "'"$5"'", "title": "'"$4"'"'"${add}" "${URL}/sendVenue"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3" "$4" "$5" "$6"
 }
 
 
+#
+# other send message variants ---------------------------------
+#
+
 # $1 CHAT $2 from chat  $3 from msg id
 forward_message() {
 	[ -z "$3" ] && return
-	sendJson "${1}" '"from_chat_id": '"${2}"', "message_id": '"${3}"'' "$FORWARD_URL"
+	sendJson "$1" '"from_chat_id": '"$2"', "message_id": '"$3"'' "${URL}/forwardMessage"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
 }
-forward() { # backward compatibility
-	forward_message "$@" || return
+
+# $1 CHAT $2 from chat  $3 from msg id
+copy_message() {
+	[ -z "$3" ] && return
+	sendJson "$1" '"from_chat_id": '"$2"', "message_id": '"$3"'' "${URL}/copyMessage"
+	# func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+	[ -n "${BOTSENT[ERROR]}" ] && processError "${FUNCNAME[0]}" "${BOTSENT[ERROR]}" "$1" "" "${BOTSENT[DESCRIPTION]}" "$2" "$3"
 }
 
 # $1 CHAT $2 bashbot formatted message, see manual advanced usage
 send_message() {
 	[ -z "$2" ] && return
 	local text keyboard btext burl no_keyboard file lat long title address sent
-	text="$(sed <<< "${2}" 's/ mykeyboardend.*//;s/ *my[kfltab][a-z]\{2,13\}startshere.*//')$(sed <<< "${2}" -n '/mytextstartshere/ s/.*mytextstartshere//p')"
+	text="$(sed <<< "$2" 's/ mykeyboardend.*//;s/ *my[kfltab][a-z]\{2,13\}startshere.*//')$(sed <<< "$2" -n '/mytextstartshere/ s/.*mytextstartshere//p')"
 	#shellcheck disable=SC2001
-	text="$(sed <<< "${text}" 's/ *mynewlinestartshere */\\n/g')"
+	text="$(sed <<< "${text}" 's/ *mynewlinestartshere */\n/g')"
 	text="${text//$'\n'/\\n}"
 	[ "$3" != "safe" ] && {
-		no_keyboard="$(sed <<< "${2}" '/mykeyboardendshere/!d;s/.*mykeyboardendshere.*/mykeyboardendshere/')"
-		keyboard="$(sed <<< "${2}" '/mykeyboardstartshere /!d;s/.*mykeyboardstartshere *//;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
-		btext="$(sed <<< "${2}" '/mybtextstartshere /!d;s/.*mybtextstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
-		burl="$(sed <<< "${2}" '/myburlstartshere /!d;s/.*myburlstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//g;s/ *mykeyboardendshere.*//g')"
-		file="$(sed <<< "${2}" '/myfilelocationstartshere /!d;s/.*myfilelocationstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
-		lat="$(sed <<< "${2}" '/mylatstartshere /!d;s/.*mylatstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
-		long="$(sed <<< "${2}" '/mylongstartshere /!d;s/.*mylongstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
-		title="$(sed <<< "${2}" '/mytitlestartshere /!d;s/.*mytitlestartshere //;s/ *my[kfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
-		address="$(sed <<< "${2}" '/myaddressstartshere /!d;s/.*myaddressstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		no_keyboard="$(sed <<< "$2" '/mykeyboardendshere/!d;s/.*mykeyboardendshere.*/mykeyboardendshere/')"
+		keyboard="$(sed <<< "$2" '/mykeyboardstartshere /!d;s/.*mykeyboardstartshere *//;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		btext="$(sed <<< "$2" '/mybtextstartshere /!d;s/.*mybtextstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		burl="$(sed <<< "$2" '/myburlstartshere /!d;s/.*myburlstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//g;s/ *mykeyboardendshere.*//g')"
+		file="$(sed <<< "$2" '/myfile[^s]*startshere /!d;s/.*myfile[^s]*startshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		lat="$(sed <<< "$2" '/mylatstartshere /!d;s/.*mylatstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		long="$(sed <<< "$2" '/mylongstartshere /!d;s/.*mylongstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		title="$(sed <<< "$2" '/mytitlestartshere /!d;s/.*mytitlestartshere //;s/ *my[kfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
+		address="$(sed <<< "$2" '/myaddressstartshere /!d;s/.*myaddressstartshere //;s/ *my[nkfltab][a-z]\{2,13\}startshere.*//;s/ *mykeyboardendshere.*//')"
 	}
-	if [ -n "$no_keyboard" ]; then
-		remove_keyboard "$1" "$text"
+	if [ -n "${no_keyboard}" ]; then
+		remove_keyboard "$1" "${text}"
 		sent=y
 	fi
-	if [ -n "$keyboard" ]; then
-		if [[ "$keyboard" != *"["* ]]; then # pre 0.60 style
+	if [ -n "${keyboard}" ]; then
+		if [[ "${keyboard}" != *"["* ]]; then	# pre 0.60 style
 			keyboard="[ ${keyboard//\" \"/\" \] , \[ \"} ]"
 		fi
-		send_keyboard "$1" "$text" "$keyboard"
+		send_keyboard "$1" "${text}" "${keyboard}"
 		sent=y
 	fi
-	if [ -n "$btext" ] && [ -n "$burl" ]; then
-		send_button "$1" "$text" "$btext" "$burl"
+	if [ -n "${btext}" ] && [ -n "${burl}" ]; then
+		send_button "$1" "${text}" "${btext}" "${burl}"
 		sent=y
 	fi
-	if [ -n "$file" ]; then
-		send_file "$1" "$file" "$text"
+	if [ -n "${file}" ]; then
+		send_file "$1" "${file}" "${text}"
 		sent=y
 	fi
-	if [ -n "$lat" ] && [ -n "$long" ]; then
-		if [ -n "$address" ] && [ -n "$title" ]; then
-			send_venue "$1" "$lat" "$long" "$title" "$address"
+	if [ -n "${lat}" ] && [ -n "${long}" ]; then
+		if [ -n "${address}" ] && [ -n "${title}" ]; then
+			send_venue "$1" "${lat}" "${long}" "${title}" "${address}"
 		else
-			send_location "$1" "$lat" "$long"
+			send_location "$1" "${lat}" "${long}"
 		fi
 		sent=y
 	fi
-	if [ "$sent" != "y" ];then
-		send_text "$1" "$text"
+	if [ "${sent}" != "y" ];then
+		send_text_mode "$1" "${text}"
 	fi
 
 }
 
 # $1 CHAT $2 message starting possibly with html_parse_mode or markdown_parse_mode
 # not working, fix or remove after 1.0!!
-send_text() {
+send_text_mode() {
 	case "$2" in
 		'html_parse_mode'*)
 			send_html_message "$1" "${2//html_parse_mode}"

mycommands.conf

@@ -0,0 +1,112 @@
+#!/bin/bash
+# shellcheck disable=SC2034
+#######################################################
+#
+#        File: mycommands.conf
+#
+# Description: place your config and messages here
+#
+#       Usage: will be sourced from mycommands.sh 
+#
+#     License: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#      Author: KayM (gnadelwartz), kay@rrr.de
+#     Created: 09.01.2021 07:27
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#######################################################
+
+##########
+# adjust your language setting here, default is C.UTF-8
+# https://github.com/topkecleon/telegram-bot-bash#setting-up-your-environment
+export 'LC_ALL=C.UTF-8'
+export 'LANG=C.UTF-8'
+export 'LANGUAGE=C.UTF-8'
+
+##########
+# in UTF-8 äöü etc. are part of [:alnum:] and ranges (e.g. a-z)
+# for more information see  doc/4_expert.md#Character_classes
+# uncomment next line if you want classic ASCII ranges for [a-z] etc.
+#export LC_COLLATE=C
+
+
+##########
+# edit the following lines to fit your bot usage
+# use ${ME} for current bot name in messages
+# Note: you must escape '_' in botname with two \ in markdown messages!
+
+# output of /info  command
+export bashbot_info='This is @'"${ME//_/\\\\_}"', the Telegram example bot written entirely in bash.
+Edit commands and messages in mycommands.sh!
+'
+
+# output of /help command (uncomment the next 2 lines
+# export bashbot_help='*Available commands*:
+# '
+
+# Set INLINE to 1 in order to receive inline queries.
+# To enable this option in your bot, send the /setinline command to @BotFather.
+export INLINE="0"
+
+# Set CALLBACK to 1 in order to receive callback queries.
+# callbacks are sent from inline_keyboards (buttons) attached tp bot messages
+export CALLBACK="0"
+
+# if your bot is group admin it get commands sent to other bots
+# Set MEONLY to 1 to ignore commands sent to other bots
+export MEONLY="0"
+
+# Set to .* to allow sending files from all locations
+# NOTE: this is a regex, not shell globbing! you must use a valid egex,
+# '.' matches any character and '.*' matches all remaining charatcers!
+# additionally you must escape special characters with '\', e.g. '\. \? \[ \*" to match them literally
+export FILE_REGEX="${BASHBOT_ETC}/.*"
+
+# set BASHBOT_RETRY to enable retry in case of recoverable errors, e.g.  throtteling
+# problems with send_xxx message etc are looged to  logs/ERROR.log
+unset BASHBOT_RETRY
+#export BASHBOT_RETRY="yes"
+
+# set value for adaptive sleeping while waiting for uodates in millisconds
+# max slepp between polling updates 10s (default 5s)
+# export BASHBOT_SLEEP="10000"
+
+# max slepp between polling updates 2s (default 5s)
+# export BASHBOT_SLEEP="2000"
+
+# add 0.2s if no update available, up to BASHBOT_SLEEP (default 0.1s)
+export BASHBOT_SLEEP_STEP="200"
+
+# if you want to use timer functions, set BASHBOT_START_TIMER to a not empty value
+# default is to not start timer
+unset BASHBOT_START_TIMER
+#export BASHBOT_START_TIMER="yes"
+
+# set to "yes" and give your bot admin privilegs to remove service messages from groups
+export SILENCER="no"
+
+# uncomment to remove keyboards sent from your bot
+# export REMOVEKEYBOARD="yes"
+# export REMOVEKEYBOARD_PRIVATE="yes"
+
+# uncomment to say welcome to new chat members
+# export WELCOME_NEWMEMBER="yes"
+WELCOME_MSG="Welcome"
+
+# uncomment to be informed about new/left chat members
+# export REPORT_NEWMEMBER="yes"
+# export REPORT_LEFTMEMBER="yes"
+
+# uncomment to send user blocked by bot a warning if they send commands
+# export NOTIFY_BLOCKED_USERS="yes"
+
+# messages for admin only commands
+NOTADMIN="Sorry, this command is allowed for admin or owner only"
+NOTBOTADMIN="Sorry, this command is allowed for bot owner only"
+
+########
+# special network setup may require additional ARGS to curl
+#
+# example: run bashbot over TOR or SOCKS proxy
+# export BASHBOT_CURL_ARGS="--socks5-hostname 127.0.0.1:9050" # TOR
+# export BASHBOT_CURL_ARGS="--socks5-hostname 127.0.0.1" # regular SOCKS
+

mycommands.sh

@@ -1,77 +1,30 @@
 #!/bin/bash
-#########
+#######################################################
 #
-# files: mycommands.sh.dist
+#        File: mycommands.sh.dist
 #
 # this is an out of the box test and example file to show what's possible in mycommands.sh
 #
 # #### if you start to develop your own bot, use the clean version of this file:
 # #### mycommands.clean
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+#       Usage: will be executed when a bot command is received 
 #
-
-# uncomment the following lines to overwrite info and help messages
-# use ${ME} for current bot name in messages
-# Note: you must escape '_' in botname with two \ in markdown messages!
-export bashbot_info='This is @'"${ME//_/\\\\_}"', the Telegram example bot written entirely in bash.
-Edit commands and messages in mycommands.sh!
-'
-# export bashbot_help='*Available commands*:
-#'
-export res=""
-
-# Set INLINE to 1 in order to receive inline queries.
-# To enable this option in your bot, send the /setinline command to @BotFather.
-export INLINE="0"
-# Set to .* to allow sending files from all locations
-# NOTE: this is a regex, not shell globbing! you must use a valid egex,
-# '.' matches any character and '.*' matches all remaining charatcers!
-# additionally you must escape special characters with '\', e.g. '\. \? \[ \*" to match them literally
-export FILE_REGEX="${BASHBOT_ETC}/.*"
-
-# set BASHBOT_RETRY to enable retry in case of recoverable errors, e.g.  throtteling
-# problems with send_xxx message etc are looged to  logs/ERROR.log
-unset BASHBOT_RETRY
-#export BASHBOT_RETRY="yes"
-
-# set value for adaptive sleeping while waiting for uodates in millisconds
-# max slepp between polling updates 10s (default 5s)
-export BASHBOT_SLEEP="10000"
-# add 0.2s if no update available, up to BASHBOT_SLEEP (default 0.1s)
-export BASHBOT_SLEEP_STEP="200"
-
-# if you want to use timer functions, set BASHBOT_START_TIMER to a not empty value
-# default is to not start timer
-unset BASHBOT_START_TIMER
-#export BASHBOT_START_TIMER="yes"
-
-# set to "yes" and give your bot admin privilegs to remove service messages from groups
-export SILENCER="no"
-
-# uncomment to remove keyboards sent from your bot
-# export REMOVEKEYBOARD="yes"
-# export REMOVEKEYBOARD_PRIVATE="yes"
-
-# uncomment to say welcome to new chat members
-# export WELCOME_NEWMEMBER="yes"
-WELCOME_MSG="Welcome"
-
-# uncomment to be informed about new/left chat members
-# export REPORT_NEWMEMBER="yes"
-# export REPORT_LEFTMEMBER="yes"
-
-# messages for admin only commands
-NOTADMIN="Sorry, this command is allowed for admin or owner only"
-NOTBOTADMIN="Sorry, this command is allowed for bot owner only"
-
-########
-# special network setup may require additional ARGS to curl
+#     License: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#      Author: KayM (gnadelwartz), kay@rrr.de
 #
-# example: run bashbot over TOR or SOCKS proxy
-# export BASHBOT_CURL_ARGS="--socks5-hostname 127.0.0.1:9050" # TOR
-# export BASHBOT_CURL_ARGS="--socks5-hostname 127.0.0.1" # regular SOCKS
+#### $$VERSION$$ v1.52-1-g0dae2db
+#######################################################
+# shellcheck disable=SC1117
 
+####################
+# Config has moved to bashbot.conf
+# shellcheck source=./commands.sh
+[ -r "${BASHBOT_ETC:-.}/mycommands.conf" ] && source "${BASHBOT_ETC:-.}/mycommands.conf"  "$1"
+
+
+##################
+# let's go ...
 if [ "$1" = "startbot" ];then
     ###################
     # this section is processed on startup
@@ -111,13 +64,13 @@ else
 
 	# example for actions based on chat or sender
 	case "${USER[ID]}+${CHAT[ID]}" in
-		'USERID+'*) # do something for all messages from USER
+		'USERID1+'*) # do something for all messages from USER
 			printf "%s: U=%s C=%s M=%s\n" "$(date)" "${USER[ID]}" "${CHAT[ID]}" "${MESSAGE}" >>"${DATADIR}/${USER[ID]}.log"
 			;;&
-		*'+CHATID') # do something for all messages from CHAT
+		*'+CHATID1') # do something for all messages from CHAT
 			printf "%s: U=%s C=%s M=%s\n" "$(date)" "${USER[ID]}" "${CHAT[ID]}" "${MESSAGE}" >>"${DATADIR}/${CHAT[ID]}.log"
 			;;&
-		'USERID+CHATID') # do something only for messages form USER in CHAT
+		'USERID2+CHATID2') # do something only for messages form USER in CHAT
 			printf "%s: U=%s C=%s M=%s\n" "$(date)" "${USER[ID]}" "${CHAT[ID]}" "${MESSAGE}" >>"${DATADIR}/${CHAT[ID]}+${USER[ID]}.log"
 			;;&
 	esac
@@ -138,7 +91,7 @@ else
 		'/echo'*) 
 			send_action "${CHAT[ID]}" "typing"
 			if ! user_is_botadmin "${USER[ID]}" ; then
-			    send_markdown_message "${CHAT[ID]}" "*${NOTBOTADMIN}*"; return 1
+			    send_markdownv2_message "${CHAT[ID]}" "*${NOTBOTADMIN}*"; return 1
 			fi
 			;;
 		# will we process edited messages also?
@@ -148,55 +101,90 @@ else
 			MESSAGE="${MESSAGE#/* }"
 			;;
 		'/_new_chat_member'*)
-			if [[ -n "${WELCOME_NEWMEMBER}" && "${NEWMEMBER[ISBOT]}" != "true" ]]; then
+			if [[ -n "${WELCOME_NEWMEMBER}" && "${NEWMEMBER[ISBOT]}" != "true" ]] && bot_is_admin "${CHAT[ID]}"; then
 			    send_normal_message "${CHAT[ID]}"\
 				"${WELCOME_MSG} ${NEWMEMBER[FIRST_NAME]} ${NEWMEMBER[LAST_NAME]} (@${NEWMEMBER[USERNAME]})"
 			    MYSENTID="${BOTSENT[ID]}"
 			    { sleep 5; delete_message  "${CHAT[ID]}" "${MYSENTID}"; } &
-			[ -n "${REPORT_NEWMEMBER}" ] && send_normal_message "$(getConfigKey "botadmin")"\
+			[ -n "${REPORT_NEWMEMBER}" ] && send_normal_message "${BOTADMIN}"\
 			    "New member: ${CHAT[TITLE]} (${CHAT[ID]}): ${NEWMEMBER[FIRST_NAME]} ${NEWMEMBER[LAST_NAME]} (@${NEWMEMBER[USERNAME]})"
 			fi
 			;;
 		'/_left_chat_member'*)
-			[ -n "${REPORT_LEFTMEMBER}" ] && send_normal_message "$(getConfigKey "botadmin")"\
+			[ -n "${REPORT_LEFTMEMBER}" ] && send_normal_message "${BOTADMIN}"\
 			    "Left member: ${CHAT[TITLE]} (${CHAT[ID]}): ${LEFTMEMBER[FIRST_NAME]} ${LEFTMEMBER[LAST_NAME]} (@${LEFTMEMBER[USERNAME]})"
 			;;
+		'/_migrate_group'*)
+			# call group migration function if provided
+			_exec_if_function my_migrate_group "${MIGRATE[FROM]}" "${MIGRATE[TO]}"
+			;;
+		
 	esac
 
 	case "${MESSAGE}" in
 		##################
-		# example commands, replace thm by your own
+		# example commands, replace them by your own
+		'/_dice_re'*) # dice from user received
+			sleep 5
+			local gameresult="*Congratulation ${USER[FIRST_NAME]} ${USER[LAST_NAME]}* you got *${MESSAGE[RESULT]} Points*."
+			[ -z "${FORWARD[UID]}" ] && send_markdownv2_message "${CHAT[ID]}" "${gameresult}"
+			;;
+		'/game'*) # send random dice, edit list to fit your needs
+			send_dice "${CHAT[ID]}" ":$(printf "slot_machine\ngame_die\ndart\nbasketball\nsoccer\nslot_machine"|sort -R|shuf -n 1shuf -n 1):"
+			if [ "${BOTSENT[OK]}" = "true" ]; then
+				local gameresult="*Congratulation ${USER[FIRST_NAME]}* ${USER[LAST_NAME]} you got *${BOTSENT[RESULT]} Points*."
+				sleep 5
+				case "${BOTSENT[RESULT]}" in
+				  1)	gameresult="*Sorry* only *one Point* ...";;
+				  2)	gameresult="*Hey*, 2 Points are *more then one!*";;
+				  5|6)	[[ "${BOTSENT[EMOJI]}" =~ fb0$ ]] || gameresult="*Super! ${BOTSENT[RESULT]} Points!*";;
+				  6*)	gameresult="*JACKPOT! ${BOTSENT[RESULT]} Points!*";;
+				esac
+				send_markdownv2_message "${CHAT[ID]}" "${gameresult}"
+			fi
+			;;
+		'/unpin'*) # unpin all messages if (bot)admin or allowed for user
+			 user_is_allowed "${USER[ID]}" "unpin" "${CHAT[ID]}" &&\
+				unpinall_chat_messages "${CHAT[ID]}"
+			;;
 		'/echo'*) # example echo command
-			send_normal_message "${CHAT[ID]}" "$MESSAGE"
+			send_normal_message "${CHAT[ID]}" "${MESSAGE}"
+			;;
+		'/button'*)# inline button, set CALLBACK=1 for processing callbacks
+			send_inline_buttons "${CHAT[ID]}" "Press Button ..." "   Button   |RANDOM-BUTTON"
 			;;
 		'/question'*) # start interactive questions
 			checkproc 
-			if [ "$res" -gt 0 ] ; then
-				startproc "examples/question.sh" || _message "Can't start question."
+			if [ "${res}" -gt 0 ] ; then
+				startproc "examples/question.sh" || send_normal_message "${CHAT[ID]}" "Can't start question."
 			else
-				send_normal_message "${CHAT[ID]}" "$MESSAGE already running ..."
+				send_normal_message "${CHAT[ID]}" "${MESSAGE} already running ..."
 			fi
 			;;
 
 		'/cancel'*) # cancel interactive command
 			checkproc
-			if [ "$res" -gt 0 ] ;then killproc && _message "Command canceled.";else _message "No command is currently running.";fi
+			if [ "${res}" -gt 0 ] ;then 
+				killproc && send_normal_message "${CHAT[ID]}" "Command canceled."
+			else
+				send_normal_message "${CHAT[ID]}" "No command is currently running."
+			fi
 			;;
 		'/run_notify'*) # start notify background job
-			myback="notify"; checkback "$myback"
-			if [ "$res" -gt 0 ] ; then
-				background "examples/notify.sh 60" "$myback" || _message "Can't start notify."
+			myback="notify"; checkback "${myback}"
+			if [ "${res}" -gt 0 ] ; then
+				background "examples/notify.sh 60" "${myback}" || send_normal_message "${CHAT[ID]}" "Can't start notify."
 			else
-				send_normal_message "${CHAT[ID]}" "Background command $myback already running ..."
+				send_normal_message "${CHAT[ID]}" "Background command ${myback} already running ..."
 			fi
 			;;
 		'/stop_notify'*) # kill notify background job
-			myback="notify"; checkback "$myback"
-			if [ "$res" -eq 0 ] ; then
-				killback "$myback"
-				send_normal_message "${CHAT[ID]}" "Background command $myback canceled."
+			myback="notify"; checkback "${myback}"
+			if [ "${res}" -eq 0 ] ; then
+				killback "${myback}"
+				send_normal_message "${CHAT[ID]}" "Background command ${myback} canceled."
 			else
-				send_normal_message "${CHAT[ID]}" "No background command $myback is currently running.."
+				send_normal_message "${CHAT[ID]}" "No background command ${myback} is currently running.."
 			fi
 			;;
 
@@ -207,12 +195,53 @@ else
 			return 0
 			;;
 		'/kickme'*) # this will replace the /kickme command
-			send_markdown_mesage "${CHAT[ID]}" "*This bot will not kick you!*"
+			send_markdownv2_mesage "${CHAT[ID]}" "This bot will *not* kick you!"
 			return 1
 			;;
 	esac
      }
 
+     mycallbacks() {
+	#######################
+	# callbacks from buttons attached to messages will be  processed here
+	# no standard use case for processing callbacks, let's log them for some users and chats
+	case "${iBUTTON[USER_ID]}+${iBUTTON[CHAT_ID]}" in
+	    'USERID1+'*) # do something for all callbacks from USER
+		printf "%s: U=%s C=%s D=%s\n" "$(date)" "${iBUTTON[USER_ID]}" "${iBUTTON[CHAT_ID]}" "${iBUTTON[DATA]}"\
+				>>"${DATADIR}/${iBUTTON[USER_ID]}.log"
+		answer_callback_query "${iBUTTON[ID]}" "Request has been logged in your user log..."
+		return
+		;;
+	    *'+CHATID1') # do something for all callbacks from CHAT
+		printf "%s: U=%s C=%s D=%s\n" "$(date)" "${iBUTTON[USER_ID]}" "${iBUTTON[CHAT_ID]}" "${iBUTTON[DATA]}"\
+				>>"${DATADIR}/${iBUTTON[CHAT_ID]}.log"
+		answer_callback_query "${iBUTTON[ID]}" "Request has been logged in chat log..."
+		return
+		;;
+	    'USERID2+CHATID2') # do something only for callbacks form USER in CHAT
+		printf "%s: U=%s C=%s D=%s\n" "$(date)" "${iBUTTON[USER_ID]}" "${iBUTTON[CHAT_ID]}" "${iBUTTON[DATA]}"\
+				>>"${DATADIR}/${iBUTTON[USER_ID]}-${iBUTTON[CHAT_ID]}.log"
+		answer_callback_query "${iBUTTON[ID]}" "Request has been logged in user-chat log..."
+		return
+		;;
+	    *)	# all other callbacks are processed here
+		local callback_answer
+		# your processing here ...
+		# message available?
+		if [[ -n "${iBUTTON[CHAT_ID]}" && -n "${iBUTTON[MESSAGE_ID]}" ]]; then
+			if [ "${iBUTTON[DATA]}" = "RANDOM-BUTTON" ]; then
+			    callback_answer="Button pressed"
+			    edit_inline_buttons "${iBUTTON[CHAT_ID]}" "${iBUTTON[MESSAGE_ID]}" "Button ${RANDOM}|RANDOM-BUTTON"
+			fi
+		else
+			    callback_answer="Button to old, sorry."
+		fi
+		# Telegram needs an ack each callback query, default empty
+		answer_callback_query "${iBUTTON[ID]}" "${callback_answer}"
+		;;
+	esac
+     }
+
      myinlines() {
 	#######################
 	# Inline query examples, do not use them in production (except image search ;-)
@@ -232,15 +261,15 @@ else
 			;;
 		"2"*)	# two photos
 			answer_inline_multi "${iQUERY[ID]}" "
-			    $(inline_query_compose "$RANDOM" "photo" "https://avatars.githubusercontent.com/u/13046303"), 
-			    $(inline_query_compose "$RANDOM" "photo" "https://avatars.githubusercontent.com/u/4593242")
+			    $(inline_query_compose "${RANDOM}" "photo" "https://avatars.githubusercontent.com/u/13046303"), 
+			    $(inline_query_compose "${RANDOM}" "photo" "https://avatars.githubusercontent.com/u/4593242")
 			    "
 			;;
 		"3"*) # three photos
 			answer_inline_multi "${iQUERY[ID]}" "
-			    $(inline_query_compose "$RANDOM" "photo" "https://avatars.githubusercontent.com/u/13046303"), 
-			    $(inline_query_compose "$RANDOM" "photo" "https://avatars.githubusercontent.com/u/4593242")
-			    $(inline_query_compose "$RANDOM" "photo" "https://avatars.githubusercontent.com/u/102707")
+			    $(inline_query_compose "${RANDOM}" "photo" "https://avatars.githubusercontent.com/u/13046303"), 
+			    $(inline_query_compose "${RANDOM}" "photo" "https://avatars.githubusercontent.com/u/4593242")
+			    $(inline_query_compose "${RANDOM}" "photo" "https://avatars.githubusercontent.com/u/102707")
 			    "
 			;;
 
@@ -249,7 +278,7 @@ else
 			local avatar=("https://avatars.githubusercontent.com/u/13046303" "https://avatars.githubusercontent.com/u/4593242" "https://avatars.githubusercontent.com/u/102707" "https://avatars.githubusercontent.com/u/6460407")
 			answer_inline_multi "${iQUERY[ID]}" "
 				$(for photo in  ${avatar[*]} ; do
-					echo "${sep}"; inline_query_compose "$RANDOM" "photo" "${photo}" "${photo}"; sep=","
+					printf "%s\n" "${sep}"; inline_query_compose "${RANDOM}" "photo" "${photo}" "${photo}"; sep=","
 				done)
 				"
 			;;
@@ -268,30 +297,63 @@ else
     # $1 current date, $2 from where the function was called, $3 ... $n optional information
     my_debug_checks() {
 	# example check because my bot created a wrong file
-	[ -f ".jssh" ] && printf "%s: %s\n" "${1}" "Ups, found file \"${PWD:-.}/.jssh\"! =========="
+	[ -f ".jssh" ] && printf "%s: %s\n" "$1" "Ups, found file \"${PWD:-.}/.jssh\"! =========="
     }
 
+    ###########################
+    # example recover from telegram block function
     # called when bashbot send_xxx command failed because we can not connect to telegram
     # return 0 to retry, return non 0 to give up
     bashbotBlockRecover() {
 	# place your commands to unblock here, e.g. change IP or simply wait
-	sleep 60 && return 0 # may be temporary
+	sleep 60 # may be temporary
+	# check connection working
+	[ -n "$(getJson "${ME_URL}")" ] && return 0
 	return 1 
     }
 
-    # place your processing functions here
+    ###########################
+    # example error processing
+    # called when delete Message failed
+    # func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+    bashbotError_delete_message() {
+	log_debug "custom errorProcessing delete_message: ERR=$2 CHAT=$3 MSGID=$6 ERTXT=$5"
+    }
+
+    # called when error 403 is returned (and no func processing)
+    # func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+    bashbotError_403() {
+	log_debug "custom errorProcessing error 403: FUNC=$1 CHAT=$3 USER=${4:-no-user} MSGID=$6 ERTXT=$5"
+	local user="$4"; [[ -z "$4" && -n "$3" ]] && user="$3"
+	if [ -n "${user}" ]; then
+		# block chat/user
+		case "$5" in
+		    *"blocked"*)
+			jssh_insertKeyDB "${user}" "User blocked bot on $(LANG=C date)" "${BLOCKEDFILE}";;
+		    *"kicked"*)
+			jssh_insertKeyDB "${user}" "Bot kicked from chat on $(LANG=C date)" "${BLOCKEDFILE}";;
+		    *)
+			jssh_insertKeyDB "${user}" "Reason: $6 on $(LANG=C date)" "${BLOCKEDFILE}";;
+		esac
+	fi
+    }
+
+
+    ###########################
+    # place your processing functions here --------------
 
     # $1 search parameter
     my_image_search(){
 	local image result sep="" count="1"
 	result="$(wget --user-agent 'Mozilla/5.0' -qO - "https://images.search.yahoo.com/search/images?p=$1" |  sed 's/</\n</g' | grep "<img src=")"
 	while read -r image; do
-		[ "$count" -gt "20" ] && break
+		[ "${count}" -gt "20" ] && break
 		image="${image#* src=\'}"; image="${image%%&pid=*}"
 		[[ "${image}" = *"src="* ]] && continue
-		echo "${sep}"; inline_query_compose "$RANDOM" "photo" "${image}"; sep=","
+		printf "%s\n" "${sep}"; inline_query_compose "${RANDOM}" "photo" "${image}"; sep=","
 		count=$(( count + 1 ))
 	done <<<"${result}"
     }
 
 fi
+

mycommands.sh.clean

@@ -1,62 +1,31 @@
 #!/bin/bash
-########
+#######################################################
+#
+#        File: mycommands.sh.clean
 #
-# files: mycommands.sh.clean
 # copy to mycommands.sh and add all your commands and functions here ...
 #
-#### $$VERSION$$ v1.1-0-gc0eb399
+#       Usage: will be executed when a bot command is received 
 #
-
-##########
-# edit the following lines to fit your bot usage
-
-export bashbot_info='This is bashbot, the Telegram bot written entirely in bash.
-'
-
-export bashbot_help='*Available commands*:
-'
-export res=""
-
-# Set INLINE to 1 in order to receive inline queries.
-# To enable this option in your bot must also send the /setinline command to @BotFather.
-export INLINE="0"
-
-# NOTE: this is a regex, not shell globbing! you must use a valid egex,
-# '.' matches any character and '.*' matches all remaining charatcers!
-# additionally you must escape special characters with '\', e.g. '\. \? \[ \*" to match them literally
-# do NOT set to .* as this allow sending files from all locations!
-export FILE_REGEX="${BASHBOT_ETC}/.*"
-
-# set BASHBOT_RETRY to enable retry in case of recoverable errors, e.g.  throtteling
-# problems with send_,´message etc are looged to  logs/ERROR.log
-unset BASHBOT_RETRY
-#export BASHBOT_RETRY="yes"
-
-# set value for adaptive sleeping while waitingnfor uodates in millisconds
-# max slepp between polling updates 10s (default 5s)
-export BASHBOT_SLEEP="10000"
-# add 0.2s if no update available, up to BASHBOT_SLEEP (default 0.1s)
-export BASHBOT_SLEEP_STEP="200"
-
-# if you want to use timer functions, set BASHBOT_START_TIMER to a not empty value
-# default is to nit start timer
-unset BASHBOT_START_TIMER
-#export BASHBOT_START_TIMER="yes"
-# set to "yes" and give your bot admin privilegs to remove service messaes from groups
-export SILENCER="no"
-
-# uncomment to remove keyboards sent by your bot
-# export REMOVEKEYBOARD="yes"
-# export REMOVEKEYBOARD_PRIVATE="yes"
-
-########
-# special network setup may require additional ARGS to curl
+#     License: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#      Author: KayM (gnadelwartz), kay@rrr.de
 #
-# example: run bashbot over TOR or SOCKS proxy
-# export BASHBOT_CURL_ARGS="--socks5-hostname 127.0.0.1:9050" # TOR
-# export BASHBOT_CURL_ARGS="--socks5-hostname 127.0.0.1" # regular SOCKS
+#### $$VERSION$$ v1.52-1-g0dae2db
+#######################################################
+# shellcheck disable=SC1117
 
+####################
+# Config has moved to bashbot.conf
+# shellcheck source=./commands.sh
+[ -r "${BASHBOT_ETC:-.}/mycommands.conf" ] && source "${BASHBOT_ETC:-.}/mycommands.conf"  "$1"
+
+
+##################
+# lets's go
 if [ "$1" = "startbot" ];then
+    ###################
+    # this section is processed on startup
+
     # run once after startup when the first message is received
     my_startup(){
 	:
@@ -86,11 +55,13 @@ else
 	[ -n "${REMOVEKEYBOARD}" ] && remove_keyboard "${CHAT[ID]}" &
 	[[ -n "${REMOVEKEYBOARD_PRIVATE}" &&  "${CHAT[ID]}" == "${USER[ID]}" ]] && remove_keyboard "${CHAT[ID]}" &
 
+	# uncommet to fix first letter upper case because of smartphone auto correction
+	#[[ "${MESSAGE}" =~  ^/[[:upper:]] ]] && MESSAGE="${MESSAGE:0:1}$(tr '[:upper:]' '[:lower:]' <<<"${MESSAGE:1:1}")${MESSAGE:2}"
 	case "${MESSAGE}" in
 		##################
 		# example command, replace them by your own
 		'/echo'*) # example echo command
-			send_normal_message "${CHAT[ID]}" "$MESSAGE"
+			send_normal_message "${CHAT[ID]}" "${MESSAGE}"
 			;;
 
 		##########
@@ -102,12 +73,25 @@ else
 			return 0
 			;;
 		'/kickme'*) # this will replace the /kickme command
-			send_markdown_mesage "${CHAT[ID]}" "*This bot will not kick you!*"
+			send_markdownv2_mesage "${CHAT[ID]}" "This bot will *not* kick you!"
 			return 1
 			;;
 	esac
      }
 
+     mycallbacks() {
+	#######################
+	# callbacks from buttons attached to messages will be  processed here
+	case "${iBUTTON[USER_ID]}+${iBUTTON[CHAT_ID]}" in
+	    *)	# all other callbacks are processed here
+		local callback_answer
+		: # your processing here ...
+		:
+		# Telegram needs an ack each callback query, default empty
+		answer_callback_query "${iBUTTON[ID]}" "${callback_answer}"
+		;;
+	esac
+     }
      myinlines() {
 	#######################
 	# this fuinction is called only if you has set INLINE=1 !!
@@ -134,12 +118,25 @@ else
 	local image result sep="" count="1"
 	result="$(wget --user-agent 'Mozilla/5.0' -qO - "https://images.search.yahoo.com/search/images?p=$1" |  sed 's/</\n</g' | grep "<img src=")"
 	while read -r image; do
-		[ "$count" -gt "20" ] && break
+		[ "${count}" -gt "20" ] && break
 		image="${image#* src=\'}"; image="${image%%&pid=*}"
 		[[ "${image}" = *"src="* ]] && continue
-		echo "${sep}"; inline_query_compose "$RANDOM" "photo" "${image}"; sep=","
+		printf "%s\n" "${sep}"; inline_query_compose "${RANDOM}" "photo" "${image}"; sep=","
 		count=$(( count + 1 ))
 	done <<<"${result}"
     }
 
+    ###########################
+    # example error processing
+    # called when delete Message failed
+    # func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+    bashbotError_delete_message() {
+	log_debug "custom errorProcessing delete_message: ERR=$2 CHAT=$3 MSGID=$6 ERTXT=$5"
+    }
+
+    # called when error 403 is returned (and no func processing)
+    # func="$1" err="$2" chat="$3" user="$4" emsg="$5" remaining args
+    bashbotError_403() {
+	log_debug "custom errorProcessing error 403: FUNC=$1 CHAT=$3 USER=${4:-no-user} MSGID=$6 ERTXT=$5"
+    }
 fi

scripts/interactive.sh.clean

@@ -1,12 +1,19 @@
 #!/bin/bash
-# file: interactive.sh
-# template for an interactive chat
-# test: start_proc "${CHAT[ID]}" "./scripts/interactive.sh.clean"
+########################################################################
+#
+# File: interactive.sh
+#
+# Description: template for an interactive chat
+#
+# Usgage: start_proc "${CHAT[ID]}" ./scripts/interactive.sh.clean
+#
+# Test in CLI: ./scripts/interactive.sh.clean
 #
 # This file is public domain in the USA and all free countries.
 # Elsewhere, consider it to be WTFPLv2. (wtfpl.net/txt/copying)
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#### $$VERSION$$ v1.52-1-g0dae2db
+########################################################################
 
 ######
 # parameters
@@ -31,10 +38,10 @@ INPUT="${3:-/dev/stdin}"
 # read -r variable <"${INPUT}"
 
 # example wait form user input and echo it
-echo "Enter a message:"
+printf "Enter a message:\n"
 
 read -r test <"${INPUT}"
-echo -e "Your Message: ${test}\nbye!"
+printf "%s\n" "Your Message: ${test}\nbye!"
 
 
 # your commands ends here

test/ADD-test-new.sh

@@ -1,58 +1,79 @@
 #!/usr/bin/env bash
+#===============================================================================
 #
-# ADD a new test skeleton to test dir, but does not activate test
+#          FILE: ADD-test-new.sh
+# 
+#         USAGE: ADD-test-new.sh
 #
-#### $$VERSION$$ v1.0-0-g99217c4
+#   DESCRIPTION: creates interactive a new test skeleton, but does not activate test
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 # magic to ensure that we're always inside the root of our application,
 # no matter from which directory we'll run script
 GIT_DIR=$(git rev-parse --git-dir)
 cd "${GIT_DIR}/.." || exit 1
 
-echo -ne "\\nDo your really want to create an new test for bashbot test suite? (y/N) N\\b"
+printf "\nDo your really want to create an new test for bashbot test suite? (y/N) N\b"
 read -r REALLY
 
-[ "${REALLY}" != "y" ] && echo "Aborting ..." && exit 1
+[ "${REALLY}" != "y" ] && printf "Stop ...\n\n" && exit 1
 
 # enter name
-echo -ne "\\nEnter Name for the the new test, 6+ chars, no :space: (empty to abort) abort\\b\\b\\b\\b\\b"
+printf "\nEnter Name for the the new test, 6+ chars, no :space: (empty to stop) stop\b\b\b\b"
 read -r NAME
 
-if [ "${NAME}" = "" ] || [ "${NAME}" = "" ]; then echo "Aborting ..."; exit 1; fi
+if [ "${NAME}" = "" ] || [ "${NAME}" = "" ]; then printf "Stop ...\n\n"; exit 1; fi
 
 # enter pass a-z
-echo -ne "\\nEnter PASS \"a\" to \"z\" to execute the new test,  d\\b"
+printf "\nEnter PASS \"a\" to \"z\" to execute the new test,  d\b"
 read -r PASS
 
 # pass to lower, default pass d 
 PASS="${PASS,,}" 
 [ "${PASS}" = "" ] && PASS="d"
-[ "${#PASS}" != '1' ] && echo "Sorry, PASS must exactly one character from a to z, aborting ..." && exit 1
+[ "${#PASS}" != '1' ] && printf "SORRY: PASS must exactly one character from a to z! Stop ...\n\n" && exit 1
 
 TEST="${PASS}-${NAME}-test"
 
-echo -e "  OK! You entered name \"${NAME}\" and pass \"${PASS}\".\\n"
+printf "%s\n\n" "  OK! Let's create test name \"${NAME}\" for pass \"${PASS}\"."
 
 # check if already exist
 if [ -f "test/${TEST}.sh" ] || [ -d "test/${TEST}" ]; then
-	echo "TEST EXIST ALREADY! Aborting ..."
+	printf "%s\n\n" "SORRY: Test test/${TEST}.sh already exists! Stop ..."
 	exit 1
 fi
 
-echo "The following files will be created for test \"${TEST}.sh\":"
-echo -e "   test/${TEST}.sh\\n   test/${TEST}/${TEST}.input\\n   test/${TEST}/${TEST}.result"
+printf "The following files will be created:\n"
+printf "%s\n%s\n%s\n" "   test/${TEST}.sh" "   test/${TEST}/${TEST}.input" "   test/${TEST}/${TEST}.result"
 
-echo -ne "\\nCreate the new test for bashbot test suite? (y/N) N\\b"
+printf "\nCreate the new test for bashbot test suite? (y/N) N\b"
 read -r REALLY
 
-[ "${REALLY}" != "y" ] && echo "Aborting ..." && exit 1
+[ "${REALLY}" != "y" ] && printf "Stop ...\n\n" && exit 1
 
-echo -e "    OK!\\n"
+printf "    OK!\n\n"
 
 # create files
 cat >"test/${TEST}.sh" <<EOF 
 #!/usr/bin/env bash
+#===============================================================================
+#
+#          FILE: test/${TEST}.sh
+# 
+#         USAGE: must run only from dev/all-tests.sh
+#
+#   DESCRIPTION: test ,,,
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: yourname, your@e-mail.com
+#
 #### \$\$VERSION\$\$
+#===============================================================================
 
 # include common functions and definitions
 # shellcheck source=test/ALL-tests.inc.sh
@@ -64,8 +85,8 @@ cd "\${TESTDIR}" || exit 1
 
 # source bashbot.sh function, uncomment if you want to test functions
 # shellcheck source=./bashbot.sh
-# source "\\${TESTDIR}/bashbot.sh" source
-# source "\\${TESTDIR}/commands.sh" source 
+# source "\${TESTDIR}/bashbot.sh" source
+# source "\${TESTDIR}/commands.sh" source 
 
 # start writing your tests here ...
 
@@ -77,4 +98,4 @@ touch "test/${TEST}/${TEST}.input" "test/${TEST}/${TEST}.result"
 set +f
 ls -l test/"${PASS}"-"${NAME}"-*
 
-echo "Done."
+printf "\nDone.\n"

test/ALL-tests.inc.sh

@@ -1,5 +1,18 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.0-0-g99217c4
+#===============================================================================
+#
+#          FILE: ALL-tests.inc.sh
+# 
+#         USAGE: source ALL-tests.inc.sh
+#
+#   DESCRIPTION: must be included from all tests,
+#                setup bashbot test environment and common test functions
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 # common variables
 export TESTME DIRME TESTDIR LOGFILE REFDIR TESTNAME
@@ -26,15 +39,20 @@ export SUCCESS NOSUCCESS
  NOSUCCESS="   FAILED!"
 
 # default input, reference and output files
-export  INPUTFILE REFFILE INPUTFILE2 REFFILE2 OUTPUTFILE
+export  INPUTFILE REFFILE OUTPUTFILE INPUTFILELIST
+ # shellcheck disable=SC2125
+ INPUTFILELIST="${DIRME}/${REFDIR}/${REFDIR}-"*".input"
  OUTPUTFILE="${TESTDIR}/${REFDIR}.out"
  INPUTFILE="${DIRME}/${REFDIR}/${REFDIR}.input"
  REFFILE="${DIRME}/${REFDIR}/${REFDIR}.result"
- INPUTFILE2="${DIRME}/${REFDIR}/${REFDIR}2.input"
- REFFILE2="${DIRME}/${REFDIR}/${REFDIR}2.result"
 
+# reset ENVIRONMENT
+export BASHBOT_URL TESTTOKEN BOTTOKEN BASHBOT_HOME BASHBOT_VAR BASHBOT_ETC
+BOTTOKEN=""
+BASHBOT_HOME=""
+BASHBOT_VAR=""
+BASHBOT_ETC=""
 # do not query telegram when testing
-export BASHBOT_URL TESTTOKEN
 BASHBOT_URL="https://my-json-server.typicode.com/topkecleon/telegram-bot-bash/getMe?"
 TESTTOKEN="123456789:BASHBOTTESTSCRIPTbashbottestscript_"
 
@@ -43,9 +61,9 @@ print_array() {
   local idx t
   local arrays=( "${@}" )
   for idx in "${arrays[@]}"; do
-    declare -n temp="$idx"
+    declare -n temp="${idx}"
 	for t in "${!temp[@]}"; do 
-  		printf '%s:\t%s\t%s\n' "$idx" "$t" "${temp[$t]}"
+  		printf '%s:\t%s\t%s\n' "${idx}" "${t}" "${temp[${t}]}"
 	done | sort
   done | grep -v '^USER:	0'
 }
@@ -53,19 +71,19 @@ print_array() {
 
 compare_sorted() {
 	local ret=0
-	sort -d -o "${1}.sort" "${1}"
-	sort -d -o "${2}.sort" "${2}"
-	diff -c "${1}.sort" "${2}.sort" || ret=1
-	[[ "${1}" != "${TESTDIR}"* ]] && rm -f "${1}.sort"
-	[[ "${2}" != "${TESTDIR}"* ]] && rm -f "${2}.sort"
-	return "$ret"
+	sort -d -o "$1.sort" "$1"
+	sort -d -o "$2.sort" "$2"
+	diff -c "$1.sort" "$2.sort" || ret=1
+	[[ "$1" != "${TESTDIR}"* ]] && rm -f "$1.sort"
+	[[ "$2" != "${TESTDIR}"* ]] && rm -f "$2.sort"
+	return "${ret}"
 }
 
 ######
 # lets go ...
-echo "Running ${TESTNAME#? } ..."
-echo "............................" 
-[ "${TESTDIR}" = "" ] && echo "${NOSUCCESS} not called from testsuite, exit" && exit 1
+printf "Running %s ...\n" "${TESTNAME#? }"
+printf "............................\n"
+[ "${TESTDIR}" = "" ] && printf "%s not called from testsuite, exit" "${NOSUCCESS}" && exit 1
 
 # reset env for test
 unset IFS;

test/a-commit-test.sh

@@ -1,5 +1,17 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.0-0-g99217c4
+#===============================================================================
+#
+#          FILE: b-example-test.sh
+# 
+#         USAGE: must run only from dev/all-tests.sh
+#
+#   DESCRIPTION: run pre-commit tests first 
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 ../dev/hooks/pre-commit.sh
 

test/b-example-test.sh

@@ -1,16 +1,35 @@
 #!/usr/bin/env bash
-# file: b-example-test.sh
-#### $$VERSION$$ v1.0-0-g99217c4
+#===============================================================================
+#
+#          FILE: b-example-test.sh
+# 
+#         USAGE: must run only from dev/all-tests.sh
+#
+#   DESCRIPTION: minimal test file as template
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 # include common functions and definitions
 # shellcheck source=test/ALL-tests.inc.sh
 source "./ALL-tests.inc.sh"
 
+###
+# place your tests here ....
+
+# example: test if TESTDIR contains file bashbot.sh
+printf "Check if bashbot.sh exists in %s ...\n" " ${TESTDIR}"
 if [ -f "${TESTDIR}/bashbot.sh" ]; then
-	echo "${SUCCESS} bashbot.sh exist!"
-	exit 0
+	printf "    ... bashbot.sh found!\n"
 else
-	echo "${NOSUCCESS} ${TESTDIR}/bashbot.sh missing!"
+	# stop test script if test failed
+	printf "%s\n" "${NOSUCCESS} ${TESTDIR}/bashbot.sh missing!"
 	exit 1
 fi
 
+# only if all tests was successful
+printf "%s\n" "${SUCCESS}"
+

test/c-init-test.sh

@@ -1,5 +1,17 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.0-0-g99217c4
+#===============================================================================
+#
+#          FILE: c-init-test.sh
+# 
+#         USAGE: must run only from dev/all-tests.sh
+#
+#   DESCRIPTION: test "bashbot.sh init" and sourcing bashbot.sh
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 # include common functions and definitions
 # shellcheck source=test/ALL-tests.inc.sh
@@ -7,44 +19,44 @@ source "./ALL-tests.inc.sh"
 
 TESTFILES="${TOKENFILE} ${ACLFILE} ${COUNTFILE} ${BLOCKEDFILE} ${ADMINFILE}"
 
-set -e
+#set -e
 
 # run bashbot first time with init
+printf "Run bashbot init ...\n"
 "${TESTDIR}/bashbot.sh" init >"${LOGFILE}"  <<EOF
-$TESTTOKEN
+${TESTTOKEN}
 nobody
 botadmin
+
 EOF
-echo "${SUCCESS}"
+printf "%s\n" "${SUCCESS}"
 
 # compare files with reference files
-echo "Check new files after init ..."
+printf "Check new files after init ...\n"
 export FAIL="0"
 for file in ${TESTFILES}
 do
 	ls -d "${TESTDIR}/${file}" >>"${LOGFILE}"
-	diff -q "${TESTDIR}/${file}" "${REFDIR}/${file}" >>"${LOGFILE}" || { echo "${NOSUCCESS} Fail diff ${file}!"; FAIL="1"; }
+	diff -q "${TESTDIR}/${file}" "${REFDIR}/${file}" >>"${LOGFILE}" || { printf "%s\n" "${NOSUCCESS} Fail diff ${file}!"; FAIL="1"; }
 done
 [ "${FAIL}" != "0" ] && exit "${FAIL}"
-echo "${SUCCESS}"
+printf "%s\n" "${SUCCESS}"
 
 trap exit 1 EXIT
 cd "${TESTDIR}" || exit
 
-echo "Test if $JSONSHFILE exists ..."
-[ ! -x "$JSONSHFILE" ] && { echo "${NOSUCCESS} json.sh not found"; exit 1; }
+printf "%s\n" "Test if ${JSONSHFILE} exists ..."
+[ ! -x "${JSONSHFILE}" ] && { printf "%s\n" "${NOSUCCESS} json.sh not found"; exit 1; }
 
-echo "Test Sourcing of bashbot.sh ..."
+printf "Test Sourcing of bashbot.sh ...\n"
 # shellcheck source=./bashbot.sh
 source "${TESTDIR}/bashbot.sh" source
-echo "Test Sourcing of commands.sh ..."
+
+printf "Test Sourcing of commands.sh ...\n"
 source "${TESTDIR}/commands.sh" source 
 
 trap '' EXIT
 cd "${DIRME}" || exit 1
-echo "${SUCCESS}"
 
-echo "Test bashbot.sh stat"
-cp "${REFDIR}/count.test" "${TESTDIR}/count.jssh"
-"${TESTDIR}/bashbot.sh" stats >"${TESTDIR}/stats.out"
-diff -q "${TESTDIR}/stats.out" "${REFDIR}/stats.out" >>"${LOGFILE}" || { echo "${NOSUCCESS} Fail diff stats output!"; FAIL="1"; }
+printf "%s\n" "${SUCCESS}"
+

test/d-JSON.sh-test.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.2-dev-13-g2a5d47d
+#### $$VERSION$$ v1.52-1-g0dae2db
 
 # include common functions and definitions
 # shellcheck source=test/ALL-tests.inc.sh
@@ -12,13 +12,13 @@ cd "${TESTDIR}" || exit 1
 
 # run JSON.sh with and without options
 cd "test" || exit 1
-echo "Check JSON.sh ..."
+printf "Check JSON.sh ...\n"
 JSON="../JSON.sh/JSON.sh"
 
 for i in 1 2
 do
-    [ "${i}" = "1" ] && echo "  ... JSON.sh -s -b -n"
-    [ "${i}" = "2" ] && echo "  ... JSON.sh"
+    [ "${i}" = "1" ] && printf "  ... JSON.sh -b -n\n"
+    [ "${i}" = "2" ] && printf "  ... JSON.sh\n"
     set +f
     for jsonfile in "${REFDIR}"/*.in
     do
@@ -29,7 +29,7 @@ do
 	# output processed input
 	diff -c "${jsonfile%.in}.result-${i}" "${jsonfile}.out-${i}" || exit 1
     done
-    echo "${SUCCESS}"
+    printf "%s\n" "${SUCCESS}"
 done
 
 cd "${DIRME}" || exit 1

test/d-process_inline-test.sh

@@ -1,5 +1,17 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.0-0-g99217c4
+#===============================================================================
+#
+#          FILE: d-process_inline-test.sh
+# 
+#         USAGE: must run only from dev/all-tests.sh
+#
+#   DESCRIPTION: test response to inline messages
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 # include common functions and definitions
 # shellcheck source=test/ALL-tests.inc.sh
@@ -16,7 +28,7 @@ source "${TESTDIR}/modules/answerInline.sh" source
 
 # overwrite get_file for test
 get_file() {
-	echo "$1"
+	printf "%s\n" "$1"
 }
 
 # get telegram input from file
@@ -26,19 +38,15 @@ declare -A UPD
 source <( printf 'UPD=( %s )' "$(sed <<<"${UPDATE}" -E -e 's/\t/=/g' -e 's/=(true|false)/="\1"/')" )
 
 # run process_message with and without python
-echo "Check process_inline ..."
-for i in 1 2
-do
-	[ "${i}" = "1" ] && ! command -v python >/dev/null 2>&1 && continue
-	[ "${i}" = "1" ] && echo "  ... with JsonDecode Python" && unset BASHBOT_DECODE
-	[ "${i}" = "2" ] && echo "  ... with JsonDecode Bash" && export BASHBOT_DECODE="yes"
-	set -x
-	{ process_inline "0";  set +x; } >>"${LOGFILE}" 2>&1;
+printf "Check process_inline ...\n"
+printf "  ... with JsonDecode Bash\n"
+set -x
+{ process_inline_query "0";  set +x; } >>"${LOGFILE}" 2>&1;
 
-	# output processed input
-	print_array "iQUERY" >"${OUTPUTFILE}"
-	compare_sorted "${REFFILE}" "${OUTPUTFILE}" || exit 1
-	echo "${SUCCESS}"
-done
+# output processed input
+print_array "iQUERY" >"${OUTPUTFILE}"
+compare_sorted "${REFFILE}" "${OUTPUTFILE}" || exit 1
+
+printf "%s\n" "${SUCCESS}"
 
 cd "${DIRME}" || exit 1

test/d-process_message-test.sh

@@ -1,5 +1,17 @@
 #!/usr/bin/env bash
-#### $$VERSION$$ v1.0-0-g99217c4
+#===============================================================================
+#
+#          FILE: d-process_message-test.sh
+# 
+#         USAGE: must run only from dev/all-tests.sh
+#
+#   DESCRIPTION: test receiving messages
+# 
+#	LICENSE: WTFPLv2 http://www.wtfpl.net/txt/copying/
+#        AUTHOR: KayM (gnadelwartz), kay@rrr.de
+#
+#### $$VERSION$$ v1.52-1-g0dae2db
+#===============================================================================
 
 # include common functions and definitions
 # shellcheck source=test/ALL-tests.inc.sh
@@ -16,7 +28,7 @@ source "${TESTDIR}/commands.sh" source
 
 # overwrite get_file for test
 get_file() {
-	echo "$1"
+	printf "%s\n" "$1"
 }
 
 # get telegram input from file
@@ -26,34 +38,24 @@ declare -Ax UPD
 # run process_message --------------
 ARRAYS="USER CHAT REPLYTO FORWARD URLS CONTACT CAPTION LOCATION MESSAGE VENUE SERVICE NEWMEMBER LEFTMEMBER PINNED"
 
-echo "Check process_message regular message..."
+printf "Check process_message ...\n"
 
-UPDATE="$(< "${INPUTFILE}")"
-Json2Array 'UPD' <"${INPUTFILE}"
-set -x
-{ pre_process_message "0"; process_message "0";  set +x; } >>"${LOGFILE}" 2>&1;
-USER[ID]="123456789"; CHAT[ID]="123456789"
+for testfile in ${INPUTFILELIST}
+do
+	printf "  ... %s\n" "${testfile##*/}"
+	testref="${testfile%.input}.result"
+	UPDATE="$(< "${testfile}")"
+	Json2Array 'UPD' <"${testfile}"
+	set -x
+	{ pre_process_message "0"; process_message "0";  set +x; } >>"${LOGFILE}" 2>&1;
+	USER[ID]="123456789"; CHAT[ID]="123456789"
 
-# output processed input
-# shellcheck disable=SC2086
-print_array ${ARRAYS}  >"${OUTPUTFILE}"
-compare_sorted "${REFFILE}" "${OUTPUTFILE}" || exit 1
+	# output processed input
+	# shellcheck disable=SC2086
+	print_array ${ARRAYS}  >"${OUTPUTFILE}"
+	compare_sorted "${testref}" "${OUTPUTFILE}" || exit 1
+	printf "%s\n" "${SUCCESS}"
 
-# run process_message ------------
-echo "Check process_message service message..."
-
-UPDATE="$(cat "${INPUTFILE2}")"
-Json2Array 'UPD' <"${INPUTFILE2}"
-set -x
-{ pre_process_message "0"; process_message "0";  set +x; } >>"${LOGFILE}" 2>&1;
-USER[ID]="123456789"; CHAT[ID]="123456789"
-
-# output processed input
-# shellcheck disable=SC2086
-print_array ${ARRAYS}  >"${OUTPUTFILE}"
-compare_sorted "${REFFILE2}" "${OUTPUTFILE}" || exit 1
-
-
-echo "${SUCCESS}"
+done
 
 cd "${DIRME}" || exit 1

test/d-process_message-test/d-process_message-test-left_chat_member.input

@@ -0,0 +1,25 @@
+["ok"]	true
+["result",0,"update_id"]	123456789
+["result",0,"message","message_id"]	123456789
+["result",0,"message","from","id"]	123456789
+["result",0,"message","from","is_bot"]	false
+["result",0,"message","from","first_name"]	"Kay"
+["result",0,"message","from","last_name"]	"M"
+["result",0,"message","from","username"]	"Gnadelwartz"
+["result",0,"message","from","language_code"]	"de"
+["result",0,"message","chat","id"]	-123456789
+["result",0,"message","chat","title"]	"Testgruppe bot only test"
+["result",0,"message","chat","type"]	"group"
+["result",0,"message","chat","all_members_are_administrators"]	true
+["result",0,"message","date"]	1592372719
+
+["result",0,"message","left_chat_participant","id"]	123456789
+["result",0,"message","left_chat_participant","is_bot"]	false
+["result",0,"message","left_chat_participant","first_name"]	"Kay"
+["result",0,"message","left_chat_participant","last_name"]	"M"
+["result",0,"message","left_chat_participant","username"]	"Gnadelwartz"
+["result",0,"message","left_chat_member","id"]	123456789
+["result",0,"message","left_chat_member","is_bot"]	false
+["result",0,"message","left_chat_member","first_name"]	"Kay"
+["result",0,"message","left_chat_member","last_name"]	"M"
+["result",0,"message","left_chat_member","username"]	"Gnadelwartz"

test/d-process_message-test/d-process_message-test-left_chat_member.result

@@ -0,0 +1,25 @@
+USER:	FIRST_NAME	Kay
+USER:	ID	123456789
+USER:	LAST_NAME	M
+USER:	USERNAME	Gnadelwartz
+CHAT:	ALL_ADMIN	true
+CHAT:	FIRST_NAME	
+CHAT:	ID	123456789
+CHAT:	LAST_NAME	
+CHAT:	TITLE	Testgruppe bot only test
+CHAT:	TYPE	group
+CHAT:	USERNAME	 
+CAPTION:	0	
+LOCATION:	LATITUDE	
+LOCATION:	LONGITUDE	
+MESSAGE:	0	/_left_chat_member  Kay M
+MESSAGE:	CAPTION	
+MESSAGE:	DICE	
+MESSAGE:	ID	123456789
+SERVICE:	0	yes
+SERVICE:	LEFTMEMBER	123456789
+LEFTMEMBER:	FIRST_NAME	Kay
+LEFTMEMBER:	ID	
+LEFTMEMBER:	ISBOT	false
+LEFTMEMBER:	LAST_NAME	M
+LEFTMEMBER:	USERNAME	Kay M

test/d-process_message-test/d-process_message-test-mesage.result

@@ -31,11 +31,13 @@ CONTACT:	LAST_NAME	Pannenhilfe
 CONTACT:	NUMBER	222222
 CONTACT:	USER_ID	
 CONTACT:	VCARD	BEGIN:VCARD\nVERSION:2.1\nN:Pannenhilfe;ADAC;;;\nFN:ADAC Pannenhilfe\nTEL;CELL;PREF:+49179222222\nTEL;X-Mobil:222222\nEND:VCARD
-CAPTION:	0	
+CAPTION:	0	Myproteine kein spell check
 LOCATION:	LATITUDE	49.631824
 LOCATION:	LONGITUDE	8.377072
 MESSAGE:	0	😂😝👌☺❤😕😈#⃣🌏🎉🙊🙉☕🚀✈🚂💯✔〽🔚
 MESSAGE:	ID	6541
+MESSAGE:	CAPTION	Myproteine kein spell check
+MESSAGE:	DICE	
 VENUE:	ADDRESS	Am Rhein 1
 VENUE:	FOURSQUARE	4c4321afce54e21eee980d1a
 VENUE:	LATITUDE	49.631824

test/d-process_message-test/d-process_message-test-message.input

@@ -89,3 +89,7 @@
 ["result",0,"message","voice","duration"]	2
 ["result",0,"message","voice","mime_type"]	"audio/ogg"
 ["result",0,"message","voice","file_size"]	4262
+["result",0,"message","caption"]	"Myproteine kein spell check"
+["result","dice","emoji"]	"\ud83c\udfb2"
+["result","dice","value"]	5
+

test/d-process_message-test/d-process_message-test-message.result

@@ -0,0 +1,45 @@
+USER:	FIRST_NAME	Kay
+USER:	ID	123456789
+USER:	LAST_NAME	M
+USER:	USERNAME	Gnadelwartz
+CHAT:	ALL_ADMIN	
+CHAT:	FIRST_NAME	Test
+CHAT:	ID	123456789
+CHAT:	LAST_NAME	Bot
+CHAT:	TITLE	BotTestTitle
+CHAT:	TYPE	private
+CHAT:	USERNAME	BotTest
+REPLYTO:	0	Ich bin der Deal-O-Mat Bot. Für eine Liste der Befehle sende /help
+REPLYTO:	FIRST_NAME	dealzbot
+REPLYTO:	ID	6542
+REPLYTO:	LAST_NAME	
+REPLYTO:	UID	987654321
+REPLYTO:	USERNAME	Deal_O_Mat_bot
+FORWARD:	FIRST_NAME	Kay
+FORWARD:	ID	6541
+FORWARD:	LAST_NAME	M
+FORWARD:	UID	123456789
+FORWARD:	USERNAME	Gnadelwartz
+URLS:	AUDIO	audio-AgADAgADL6oxG-Gw4EndCWGl2WUfUo1pXw8ABOusSilDGzAYa
+URLS:	DOCUMENT	document-AgADAgADL6oxG-Gw4EndCWGl2WUfUo1pXw8ABOusSilDGzAYa
+URLS:	PHOTO	photo-AgADAgADL6oxG-Gw4EndCWGl2WUfUo1pXw8ABOusSilDGzAYa
+URLS:	STICKER	sticker-AgADAgADL6oxG-Gw4EndCWGl2WUfUo1pXw8ABOusSilDGzAYa
+URLS:	VIDEO	video-AgADAgADL6oxG-Gw4EndCWGl2WUfUo1pXw8ABOusSilDGzAYa
+URLS:	VOICE	voice-AgADAgADL6oxG-Gw4EndCWGl2WUfUo1pXw8ABOusSilDGzAYa
+CONTACT:	FIRST_NAME	ADAC
+CONTACT:	LAST_NAME	Pannenhilfe
+CONTACT:	NUMBER	222222
+CONTACT:	USER_ID	
+CONTACT:	VCARD	BEGIN:VCARD\nVERSION:2.1\nN:Pannenhilfe;ADAC;;;\nFN:ADAC Pannenhilfe\nTEL;CELL;PREF:+49179222222\nTEL;X-Mobil:222222\nEND:VCARD
+CAPTION:	0	Myproteine kein spell check
+LOCATION:	LATITUDE	49.631824
+LOCATION:	LONGITUDE	8.377072
+MESSAGE:	0	😂😝👌☺❤😕😈#⃣🌏🎉🙊🙉☕🚀✈🚂💯✔〽🔚
+MESSAGE:	CAPTION	Myproteine kein spell check
+MESSAGE:	DICE	
+MESSAGE:	ID	6541
+VENUE:	ADDRESS	Am Rhein 1
+VENUE:	FOURSQUARE	4c4321afce54e21eee980d1a
+VENUE:	LATITUDE	49.631824
+VENUE:	LONGITUDE	8.377072
+VENUE:	TITLE	Kolb's Biergarten

test/d-process_message-test/d-process_message-test-new_chat_member.input

@@ -0,0 +1,25 @@
+["ok"]	true
+["result",0,"update_id"]	123456789
+["result",0,"message","message_id"]	123456789
+["result",0,"message","from","id"]	123456789
+["result",0,"message","from","is_bot"]	false
+["result",0,"message","from","first_name"]	"Kay"
+["result",0,"message","from","last_name"]	"M"
+["result",0,"message","from","username"]	"Gnadelwartz"
+["result",0,"message","from","language_code"]	"de"
+["result",0,"message","chat","id"]	-123456789
+["result",0,"message","chat","title"]	"Testgruppe bot only test"
+["result",0,"message","chat","type"]	"group"
+["result",0,"message","chat","all_members_are_administrators"]	true
+["result",0,"message","date"]	1592372719
+
+["result",0,"message","new_chat_participant","id"]	123456789
+["result",0,"message","new_chat_participant","is_bot"]	false
+["result",0,"message","new_chat_participant","first_name"]	"Kay"
+["result",0,"message","new_chat_participant","last_name"]	"M"
+["result",0,"message","new_chat_participant","username"]	"Gnadelwartz"
+["result",0,"message","new_chat_member","id"]	123456789
+["result",0,"message","new_chat_member","is_bot"]	false
+["result",0,"message","new_chat_member","first_name"]	"Kay"
+["result",0,"message","new_chat_member","last_name"]	"M"
+["result",0,"message","new_chat_member","username"]	"Gnadelwartz"

test/d-process_message-test/d-process_message-test-new_chat_member.result

@@ -13,21 +13,15 @@ CAPTION:	0
 LOCATION:	LATITUDE	
 LOCATION:	LONGITUDE	
 MESSAGE:	0	/_new_chat_member 123456789 Gnadelwartz
+MESSAGE:	CAPTION	
+MESSAGE:	DICE	
 MESSAGE:	ID	123456789
 SERVICE:	0	yes
-SERVICE:	LEFTMEMBER	123456789
 SERVICE:	NEWMEMBER	123456789
-SERVICE:	NEWPHOTO	AgACAgIAAxkBAAEBFute6a3vIpB99vim811hxeu2tyQWfwACrKwxG0TMUUtDBH10RqlzGCukupIuAAMBAAMCAANhAAM3SAMAARoE
-SERVICE:	NEWTITLE	new Testgruppe bot only
-SERVICE:	PINNED	3022
+SERVICE:	NEWPHOTO	
+SERVICE:	NEWTITLE	
 NEWMEMBER:	FIRST_NAME	Kay
 NEWMEMBER:	ID	123456789
 NEWMEMBER:	ISBOT	false
 NEWMEMBER:	LAST_NAME	M
 NEWMEMBER:	USERNAME	Gnadelwartz
-LEFTMEMBER:	FIRST_NAME	Kay
-LEFTMEMBER:	ID	
-LEFTMEMBER:	ISBOT	false
-LEFTMEMBER:	LAST_NAME	M
-PINNED:	ID	3022
-PINNED:	MESSAGE	new pinned Message

test/d-process_message-test/d-process_message-test-new_chat_picture.input

@@ -0,0 +1,30 @@
+["ok"]	true
+["result",0,"update_id"]	123456789
+["result",0,"message","message_id"]	123456789
+["result",0,"message","from","id"]	123456789
+["result",0,"message","from","is_bot"]	false
+["result",0,"message","from","first_name"]	"Kay"
+["result",0,"message","from","last_name"]	"M"
+["result",0,"message","from","username"]	"Gnadelwartz"
+["result",0,"message","from","language_code"]	"de"
+["result",0,"message","chat","id"]	-123456789
+["result",0,"message","chat","title"]	"Testgruppe bot only test"
+["result",0,"message","chat","type"]	"group"
+["result",0,"message","chat","all_members_are_administrators"]	true
+["result",0,"message","date"]	1592372719
+
+["result",0,"message","new_chat_photo",0,"file_id"]	"AgACAgIAAxkBAAEBFute6a3vIpB99vim811hxeu2tyQWfwACrKwxG0TMUUtDBH10RqlzGCukupIuAAMBAAMCAANhAAM3SAMAARoE"
+["result",0,"message","new_chat_photo",0,"file_unique_id"]	"AQADK6S6ki4AAzdIAwAB"
+["result",0,"message","new_chat_photo",0,"file_size"]	5939
+["result",0,"message","new_chat_photo",0,"width"]	160
+["result",0,"message","new_chat_photo",0,"height"]	160
+["result",0,"message","new_chat_photo",1,"file_id"]	"AgACAgIAAxkBAAEBFute6a3vIpB99vim811hxeu2tyQWfwACrKwxG0TMUUtDBH10RqlzGCukupIuAAMBAAMCAANiAAM4SAMAARoE"
+["result",0,"message","new_chat_photo",1,"file_unique_id"]	"AQADK6S6ki4AAzhIAwAB"
+["result",0,"message","new_chat_photo",1,"file_size"]	14124
+["result",0,"message","new_chat_photo",1,"width"]	320
+["result",0,"message","new_chat_photo",1,"height"]	320
+["result",0,"message","new_chat_photo",2,"file_id"]	"AgACAgIAAxkBAAEBFute6a3vIpB99vim811hxeu2tyQWfwACrKwxG0TMUUtDBH10RqlzGCukupIuAAMBAAMCAANjAAM5SAMAARoE"
+["result",0,"message","new_chat_photo",2,"file_unique_id"]	"AQADK6S6ki4AAzlIAwAB"
+["result",0,"message","new_chat_photo",2,"file_size"]	34052
+["result",0,"message","new_chat_photo",2,"width"]	640
+["result",0,"message","new_chat_photo",2,"height"]	640

test/d-process_message-test/d-process_message-test-new_chat_picture.result

@@ -0,0 +1,21 @@
+USER:	FIRST_NAME	Kay
+USER:	ID	123456789
+USER:	LAST_NAME	M
+USER:	USERNAME	Gnadelwartz
+CHAT:	ALL_ADMIN	true
+CHAT:	FIRST_NAME	
+CHAT:	ID	123456789
+CHAT:	LAST_NAME	
+CHAT:	TITLE	Testgruppe bot only test
+CHAT:	TYPE	group
+CHAT:	USERNAME	 
+CAPTION:	0	
+LOCATION:	LATITUDE	
+LOCATION:	LONGITUDE	
+MESSAGE:	0	/_new_chat_photo 123456789 AgACAgIAAxkBAAEBFute6a3vIpB99vim811hxeu2tyQWfwACrKwxG0TMUUtDBH10RqlzGCukupIuAAMBAAMCAANhAAM3SAMAARoE
+MESSAGE:	CAPTION	
+MESSAGE:	DICE	
+MESSAGE:	ID	123456789
+SERVICE:	0	yes
+SERVICE:	NEWPHOTO	AgACAgIAAxkBAAEBFute6a3vIpB99vim811hxeu2tyQWfwACrKwxG0TMUUtDBH10RqlzGCukupIuAAMBAAMCAANhAAM3SAMAARoE
+SERVICE:	NEWTITLE