FREE PLUGIN: Threads Counter (UBot Threading Fixed)

Threads CounterAs you may be aware threading in UBot is not safe and may not work as expect under certain conditions, which would result in threads slowly dying out, so at the end, you only get left with 1 active thread, although you coded the bot to run with multiple.

To solve this problem I have created a plugin which will provide you with a dedicated “threads counter”, which uses atomic increment/decrement operations and will allow you to implement threading properly.

UPDATE 10/10/2014

With release of UBot v5.5 threading seems to be fixed, so the bug that Threads Counter plugin was solving in UBot v4 doesn’t exist anymore, that’s why you don’t need it anymore for advanced threading. I’m still using threads counter plugin so I can fall back to v4 whenever I have problems with v5.

THE PROBLEM EXPLAINED

The problem with counting threads lies in UBot increment and decrement commands, which are not atomic operations; that means that the operations increment (read, add 1, set) and decrement  (read, subtract one and set) are not executed in one processor cycle, which may result in wrong counting of threads, if two or more threads try to increment/decrement the same variable (value) at the same time.

Most of the threading examples found on UBot forum suffer from this problem because decrement command gets executed from inside the threads container. If you imagine that the initial value of counter is 2, and 2 threads decrement that value at the same time, they will both read the same value 2 and decrement it, which is wrong, since the final result would be 1 instead of 0. This would result in wrong thread counting , since bot would think that 1 thread is still running (although we closed it, counter is displaying it as still running) so it would never stop because of that. You would also notice manifestation of these problems before, since bot could end up running on 1 thread only (all other threads failed to decrement counter, so bot still thinks those threads are running).

To read more about the problem and test which proved threading doesn’t work as expected check this post on UBot forum.

 

THREADS COUNTER FUNCTION

Threads Counter plugin will add a new function to your UBot environment called “threads counter”, which can perform 4 different operations that you can use to change threads counter value.

THREADS COUNTER OPERATIONS

Operations should be quite self explanatory, but still, let’s put them down here:

Read

Reads a threads counter and returns its value.

Reset

Resets a threads counter to zero and returns that value.

Increment (Atomic)

Performs atomic increment on threads counter and returns it’s value.

Decrement (Atomic)

Performs atomic decrement on threads counter and returns it’s value.

 

HOW TO USE THREADS COUNTER FUNCTION

You can use threads counter function in similar fashion that you have used your previous counter, you should just avoid using UBot commands increment/decrement from threads; instead you should use different function operations to manipulate the counter value.

UBOT CODE EXAMPLES

Threads Counter Unit Test Code

The example bellow was re-coded from the code that was proved to not work correctly (you can find it here) and it only replaces UBot threads counter (operations) with the plugin ones. If you run the code bellow in UBot you will notice that with new threads counter UBot will be able to properly count threads and will also be able to stop when all threads are done (which wasn’t true with broken counter, since a few threads always remained alive in eyes of UBot).

Bellow the Main Threading Loop you can see some additional code which was used to find threading bug. The code was waiting for all threads to close (#THREADS Active) or timeout, in case UBot still saw threads opened and to alert the result.

Scraper With Proxies And Multiple Threads

Bellow is a simple example which can serve as basis for your next multithreaded project with proxies. It is programed the way that each thread gets it’s own proxy and scrapes “whatsmyip” website; it stores the information to &RESULTS table which is saved to results.csv file at the end.

 To test the code please copy paste it to UBots “code view” and save it to a file. Next save your proxies to the same folder you saved the code, and name the file “proxies.txt”. After that you are ready to run the code.

COMMAND: define THREAD START(#PROXIES ROW, #RESULTS ROW)

This is the command which spawns a new thread, opens a new browser and executes “CHANGE PROXY” and “SCRAPE WHATSMYIP” inside them. At the end, before closing the thread we let the other code know that thread got closed, taht’s why we decrement threads coutner.

COMMAND: define CHANGE PROXY(#PROXIES ROW)

Command CHANGE PROXY has one parameter called “#PROXIES ROW”, which tells it which proxy it should use/in which row the proxy is. After it gets the location it adds the proxy to a list, which looks like this:

  • Proxy IP
  • Proxy Port
  • Proxy Username (optional)
  • Proxy Password (optional)

As you can see last 2 parameters are optional, that’s why we first check if list has length 4, and if does, we use “set proxy credentials” command and change proxy after that command. If list length is not 4 we then check if it’s 2 (that’s minimum requirement), and if so we only change the proxy without providing any credentials. In case that %PROXY list length is not equal to 2 or 4 we stop the bot, since the list doesn’t contain a properly formatted proxy.

 COMMAND: define SCRAPE WHATSMYIP(#PROXIES ROW, #RESULTS ROW)

The command SCRAPE WHATSMYIP contains the code which does the real work; it first navigates to the page we are scraping and wait’s for it to load. Once the page is loaded we check if it loaded successfully (if proxy works), and if so we scrape some elements and set STATUS column (#6) to SUCCESS. If the element that we are expecting doesn’t show up it means that we were not able to load that page via proxy, that’s why we set STATUS column to FAIL.

CONCLUSION

Currently this plugin solves the threads counter problem only, but I think the problem still exist with other variables where you are using increment/decrement commands from thread (like progress counters for example); so instead of changing those counters from thread you should be changing them from the main/UI thread.

However, I think UBot team should be taking care of that problem in v5, since I think this is one of the fundamental things they should handle if they want to allow us proper threading,

UBOT FORUM DISCUSSION

[FREE PLUGIN] Threads Counter – UBot v4 Threading Fixed

 

DOWNLOAD

DOWNLOAD FREE PLUGINS

If you find this post helpful please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.

3 thoughts on “FREE PLUGIN: Threads Counter (UBot Threading Fixed)

  1. Pingback: Setting Up Threading With Status

Leave a Reply

Your email address will not be published. Required fields are marked *

*