Describe the bug Unable to install udemyscraper via pip install

To Reproduce ERROR: Cannot install udemyscraper==0.8.1 and udemyscraper==0.8.2 because these package versions have conflicting dependencies.

The conflict is caused by: udemyscraper 0.8.2 depends on getopt2==0.0.3 udemyscraper 0.8.1 depends on getopt2==0.0.3

Desktop (please complete the following information):

• OS: MAC OS

Describe the bug When running the sample code all I get is timeout.

To Reproduce Steps to reproduce the behavior: Run the sample code from udemyscraper import UdemyCourse

course = UdemyCourse() course.fetch_course('learn javascript') print(course.title)

Current behavior Timed out waiting for page to load or could not find a matching course

OS: MACOS

EXPERIMENTAL! Needs Testing.

time.sleep() introduces a necessary wait, even if the page has already been loaded.

By using expected_components, we can proceed as and when the element loads. Using the python time library, I calculated the time taken by search and course page to load to be 2 seconds (approx.)

Theoretically speaking, after the change, execution time should have reduced by 5 seconds. (3+4-2) However, the gain was only 3 seconds instead of the expected 5.

This behavior seems unexpected for the moment, unless we can find where the missing 2 seconds are. For a reference, the original version, using time.sleep() took 17 seconds to execute.

(All times are measured for my internet connection, by executing the given example in readme file)

Possibly need to dig further. I haven't yet got the time to read full code.

Here 4 seconds have been hardcoded, it will be better to wait for the search results to load and then get the source code.

A basic method to do this would be to check if search element is visible or not, once its visible, it can proceed to fetch source code. This way if you have a really fast connection, you wouldn't need to wait longer and vice-versa.

It seems that on the search page, the classes of the elements keep changing. So it would be best to only fetch the course url and then fetch all the other data from the course page itself.

Experimental!!

Export the entire dictionary to a xml file using the dict2xml library.

• [x] Make branch even with refractor base
• [x] Switch to dicttoxml from dict2xml
• [x] Object arrays of sections and lessons are not grouped under one root called Sections or Lessons. This is also the case for all of the other arrays.
• [x] Rename List item
• [x] Rename root tag to course

Setup a way to automatically fetch browser drivers based on user's choice (chromium/firefox) corresponding to the installed browser version.

The hard part will be to find the version of the browser installed.

Whenever I try to scrape a course from udemy I get this error-

on 1: Timed out waiting for page to load or could not find a matching course
Scraping Course |████████████████████████████████████████| 1 in 29.5s (0.03/s)

It was working a couple of times before but now it doesn't work..

Steps to reproduce the behavior:

1. This happens both when using the script and the module
2. I used query argument
3. Output-

Desktop (please complete the following information):

• OS: Windows 10
• Browser: Chromium
• Version: 92

I did checked by manually opening chromium and searching for the course. But when I use the scraper, it doesn't work.

Some tests have shown that it is way more efficient to use css selectors than find, especially with nested finds which tend to be wayyy more slow and time consuming. It would be better replace all of the find statements with select and then use direct path.

Instead of editing the source code to select which browser you would like to use, you can now specify the same while initializing the UdemyCourse class or by simply using an argument when using the standalone way.

    -b  --browser       Allows you to select the browser you would like to use for Scraping
                        Values: "chrome" or "firefox". Defaults to chrome if no argument is passed.

Also provided a gekodriver.exe binary.

I assume that the main udemyScraper.py file will be used as a module, so instead I made another file main.py which can be used for such operations. As of now only some basic arguments have been added. Will add more in the future.

    -h  --help          Displays information about udemyscraper and its usage
    -v  --version       Displays the version of the tool
    -n  --no-warn       Disables the warning when initializing the udemyscourse class