Bug#952733: dnsrecon fails in Google Search enumeration due to incorrect urllib functions being used

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#952733: dnsrecon fails in Google Search enumeration due to incorrect urllib functions being used

Jonas Andradas
Package: dnsrecon
Version: 0.9.1-3
Severity: normal

Dear Maintainer,

When using dnsrecon and requesting it to use the Google search (-t goo), it
fails due to an invocation to "urllib.urlopen(url)", which fails with an error
informing that there is no urlopen attribute in the module urllib, as can be
seen in the following example run.  I would like to point out that the same
command did work on February 26th 2020, and stopped working on February 28th
2020, after some apt upgrade (however, I cannot provide which packages were
updated).

$ dnsrecon -d debian.org -t goo
[*] Performing Google Search Enumeration against debian.org
Traceback (most recent call last):
  File "/usr/share/dnsrecon/lib/gooenum.py", line 54, in scrape_google
    sock = urllib.urlopen(url)
AttributeError: module 'urllib' has no attribute 'urlopen'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./dnsrecon.py", line 1788, in <module>
    main()
  File "./dnsrecon.py", line 1672, in main
    goo_enum_records = se_result_process(res, scrape_google(domain))
  File "/usr/share/dnsrecon/lib/gooenum.py", line 58, in scrape_google
    sock = urllib.request.urlopen(request)
  File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 429: Too Many Requests



-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.4.0-4-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dnsrecon depends on:
ii  python3            3.7.5-3
ii  python3-dnspython  1.16.0-1
ii  python3-lxml       4.5.0-1
ii  python3-netaddr    0.7.19-3

dnsrecon recommends no packages.

dnsrecon suggests no packages.

-- no debconf information

Reply | Threaded
Open this post in threaded view
|

Bug#952733: dnsrecon fails in Google Search enumeration due to incorrect urllib functions being used

Jonas Andradas
Package: dnsrecon
Followup-For: Bug #952733

Dear Maintainer,

new updates performed along the day fixed the issue... sorry for reporting it
too soon!



-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.4.0-4-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dnsrecon depends on:
ii  python3            3.7.5-3
ii  python3-dnspython  1.16.0-1
ii  python3-lxml       4.5.0-1
ii  python3-netaddr    0.7.19-3

dnsrecon recommends no packages.

dnsrecon suggests no packages.

-- no debconf information

Reply | Threaded
Open this post in threaded view
|

Bug#952733: dnsrecon fails in Google Search enumeration due to incorrect urllib functions being used

Raphael Hertzog-3
On Fri, 28 Feb 2020, Jonas Andradas wrote:
> new updates performed along the day fixed the issue... sorry for reporting it
> too soon!

What updates? It seems weird that this issue would fix itself.

Cheers,
--
  ⢀⣴⠾⠻⢶⣦⠀   Raphaël Hertzog <[hidden email]>
  ⣾⠁⢠⠒⠀⣿⡁
  ⢿⡄⠘⠷⠚⠋    The Debian Handbook: https://debian-handbook.info/get/
  ⠈⠳⣄⠀⠀⠀⠀   Debian Long Term Support: https://deb.li/LTS

Reply | Threaded
Open this post in threaded view
|

Bug#952733: dnsrecon fails in Google Search enumeration due to incorrect urllib functions being used

Jonas Andradas
My best guess is there was an update to one of the underlying libraries I did not notice (note that this was on Sid, so frequent updates). 

Other option would be that the error only happens if the functionality fails (e.g. Google blocking the request), so the full error trace is shown, meaning that the urllib error is not fatal, and is only shown when the functionality actually fails.

I replied to the bug because trying to reproduce it some time after raising it (and retrying now), I get the expected behavior and not the error...

On Tue, 3 Mar 2020, 09:32 Raphael Hertzog, <[hidden email]> wrote:
On Fri, 28 Feb 2020, Jonas Andradas wrote:
> new updates performed along the day fixed the issue... sorry for reporting it
> too soon!

What updates? It seems weird that this issue would fix itself.

Cheers,
--
  ⢀⣴⠾⠻⢶⣦⠀   Raphaël Hertzog <[hidden email]>
  ⣾⠁⢠⠒⠀⣿⡁
  ⢿⡄⠘⠷⠚⠋    The Debian Handbook: https://debian-handbook.info/get/
  ⠈⠳⣄⠀⠀⠀⠀   Debian Long Term Support: https://deb.li/LTS