-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Harvester Crashes when JSON Harvester Crashes... #164
Comments
Issue Still exists |
example json:
|
i know the json is wrong for the harvester, but it shouldnt cause other harvest jobs to not be processed at all beeing stuck in "running" for months |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2019-10-10 12:28:13,045 DEBUG [ckanext.dcat.harvesters._json] In DCATJSONHarvester import_stage
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 10, in
sys.exit(run())
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
result = self.command()
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 192, in command
fetch_callback(consumer, method, header, body)
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line 418, in fetch_callback
fetch_and_import_stages(harvester, obj)
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line 436, in fetch_and_import_stages
success_import = harvester.import_stage(obj)
File "/usr/lib/ckan/default/src/ckanext-dcat/ckanext/dcat/harvesters/_json.py", line 225, in import_stage
self._get_package_name(harvest_object, package_dict['title'])
File "/usr/lib/ckan/default/src/ckanext-dcat/ckanext/dcat/harvesters/base.py", line 134, in _get_package_name
name = self._gen_new_name(title)
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/harvesters/base.py", line 90, in _gen_new_name
ideal_name = munge_title_to_name(title)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/munge.py", line 47, in munge_title_to_name
name = re.sub('[ .:/]', '-', name)
File "/usr/lib/ckan/default/lib/python2.7/re.py", line 155, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer
should instead be handled and continue on with next harvest source dont u think?
The text was updated successfully, but these errors were encountered: