Skip to content

Commit

Permalink
Merge branch 'master' of https://github.com/datahq/datahub-cli
Browse files Browse the repository at this point in the history
  • Loading branch information
anuveyatsu committed May 28, 2018
2 parents 8c3e519 + f3d5a89 commit b780ad2
Show file tree
Hide file tree
Showing 4 changed files with 53 additions and 45 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ data push # upload the dataset onto a DataHub
> "As a result you'll got a link to share:
http://datahub.io/user-name/data-package-name
```
That's it! Your data is online. You can make your data public or private, add some pretty graphics, and many more. Please read http://datahub.io/docs for details.
That's it! Your data is online. You can make your data unlisted or private, add some pretty graphics, and many more. Please read http://datahub.io/docs for details.
With `data-cli` you can also:
* Get data from online sources
Expand Down Expand Up @@ -46,7 +46,7 @@ Mydataset Mydataset
your data is published!
🔗 https://datahub.io/myname/mydataset/v/1 (copied to clipboard)
```
**Note:** by default, findability flag for your dataset is set to `--unlisted`, meaning nobody else is able to see it, except you. Use `--public` flag to make it publicly available
**Note:** by default, findability flag for your dataset is set to `--public`. Use `--unlisted` flag if you want it to not appear in the search results.
### [data get](https://github.com/datahq/data-cli/blob/master/docs/get.md)
Expand Down
12 changes: 6 additions & 6 deletions bin/data-push.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ const info = require('../lib/utils/output/info.js')

const argv = minimist(process.argv.slice(2), {
string: ['push', 'sheets'],
boolean: ['help', 'test', 'debug', 'interactive', 'published', 'private', 'zip', 'sqlite'],
boolean: ['help', 'test', 'debug', 'interactive', 'unlisted', 'private', 'zip', 'sqlite'],
alias: {help: 'h', interactive: 'i', sheets: 'sheet'}
})

Expand Down Expand Up @@ -91,9 +91,9 @@ Promise.resolve().then(async () => {
ownerid: config.get('profile') ? config.get('profile').id : config.get('id'),
owner: config.get('profile') ? config.get('profile').username : config.get('username')
}
let findability = 'unlisted'
if (argv.published || argv.publish || argv.public) {
findability = 'published'
let findability = 'published'
if (argv.unlisted) {
findability = 'unlisted'
}
if (argv.private) {
findability = 'private'
Expand Down Expand Up @@ -146,7 +146,7 @@ Promise.resolve().then(async () => {
})
}
})

const res = await datahub.push(dataset, options)
// Analytics:
if (process.env.datahub !== 'dev') {
Expand All @@ -168,7 +168,7 @@ Promise.resolve().then(async () => {
// Print success message and provide URL to showcase page:
let revisionId = res.flow_id.split('/').pop()
const message = '\n🙌 your data is published!\n'
const url = urljoin(config.get('domain'), datahubConfigs.owner, dataset.descriptor.name,'v',revisionId,'?source=cli')
const url = urljoin(config.get('domain'), datahubConfigs.owner, dataset.descriptor.name, 'v', revisionId)
let copied = ' (copied to clipboard)'
try {
await copyToClipboard(url)
Expand Down
36 changes: 18 additions & 18 deletions docs/push.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,36 @@

■ data push [PATH]

`PATH` (optional) is the path to the data file or data package.

## Options:

-h, --help Output usage information.
--format Explicitely set the format for a file. Useful when a file does not have conventional

--format Explicitly set the format for a file. Useful when a file does not have conventional
naming. E.g., `--format=csv`

-i, --interactive Enable interactive mode. Useful when pushing a single file.
--schedule Setup a schedule so the DataHub will automatically re-import the remote file on
a regular basis. E.g., `every 90s`, `every 5m`, `every 2d`. The number is always
an integer, selector is `s/m/h/d/w` (second -> week) and you can’t schedule for

--schedule Setup a schedule so the DataHub will automatically re-import the remote file on
a regular basis. E.g., `every 90s`, `every 5m`, `every 2d`. The number is always
an integer, selector is `s/m/h/d/w` (second -> week) and you can’t schedule for
less than 60 seconds.
--sheets Set which sheets should be processed when pushing Excel files. By default, only

--sheets Set which sheets should be processed when pushing Excel files. By default, only
the first sheet is processed. You can use `--sheets=all` option to push "all" sheets.
You also can list sheet numbers, e.g., `--sheets=1,2`. If you wanted to push only
You also can list sheet numbers, e.g., `--sheets=1,2`. If you wanted to push only
the second sheet, you would do `--sheets=2`. Sheet number starts from 1.

### findability options:

This options define the dataset visibility on the DataHub.io site:

--unlisted (default) Other users will not see the dataset in the search results.
You will see the dataset in the search results.
--public (default) Everybody can see the dataset in the search results.
Everybody can access the dataset by the URL link.

--public Everybody can see the dataset in the search results.
--unlisted Other users will not see the dataset in the search results.
You will see the dataset in the search results.
Everybody can access the dataset by the URL link.

--private Other users cannot access the dataset.
Expand All @@ -47,14 +47,14 @@ Uploads Data Package to DataHub with path (core/finance-vix/ should have datapac

■ data push core/finance-vix/

By default, all pushed datasets are unlisted. To make them published:
By default, all pushed datasets are public. To make them unlisted:

■ data push core/finance-vix/ --unlisted

■ data push core/finance-vix/ --published

Uploads a file from URL to DataHub on weekly basis and sets file format as CSV:

■ data push URL --schedule="every 1w" --format=csv

Uploads a Excel file and processes only the second sheet:

■ data push myExcel.xlsx --sheets=2
46 changes: 27 additions & 19 deletions test/push/push.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -256,17 +256,17 @@ test('push command for empty files: no ext, html, txt, json', async t => {
let args = '--name=empty-no-extension'
let result = await runcli('push', path_, args)
let stdout = result.stdout.split('\n')
let hasErrorMessage = stdout.find(item => item.includes('You can not push empty files, please add some data and try again'))
let hasPathToFile = stdout.find(item => item.includes('test/fixtures/test-data/files/empty-files/'))
t.truthy(hasErrorMessage)
t.truthy(hasPathToFile)
let hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
let hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/empty-no-extension/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)

path_ = 'test/fixtures/test-data/files/empty-files/empty.html'
args = '--name=empty-html'
result = await runcli('push', path_, args)
stdout = result.stdout.split('\n')
let hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
let hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/empty-html/v/'))
hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/empty-html/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)
let whatsInClipboard = await clipboardy.read()
Expand All @@ -276,10 +276,10 @@ test('push command for empty files: no ext, html, txt, json', async t => {
args = '--name=empty-txt'
result = await runcli('push', path_, args)
stdout = result.stdout.split('\n')
hasErrorMessage = stdout.find(item => item.includes('You can not push empty files, please add some data and try again'))
hasPathToFile = stdout.find(item => item.includes('test/fixtures/test-data/files/empty-files/'))
t.truthy(hasErrorMessage)
t.truthy(hasPathToFile)
hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/empty-txt/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)

path_ = 'test/fixtures/test-data/files/empty-files/empty.json'
args = '--name=empty-json'
Expand Down Expand Up @@ -318,8 +318,10 @@ test('push command fails for zero byte files', async t => {
let args = '--name=zero'
let result = await runcli('push', path_, args)
let stdout = result.stdout.split('\n')
let hasErrorMsg = stdout.find(item => item.includes('> You can not push empty files, please add some data and try again'))
t.truthy(hasErrorMsg)
let hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
let hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/zero/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)

path_ = 'test/fixtures/test-data/files/zero-files/zero.csv'
result = await runcli('push', path_, args)
Expand All @@ -329,25 +331,31 @@ test('push command fails for zero byte files', async t => {
path_ = 'test/fixtures/test-data/files/zero-files/zero.html'
result = await runcli('push', path_, args)
stdout = result.stdout.split('\n')
hasErrorMsg = stdout.find(item => item.includes('> You can not push empty files, please add some data and try again'))
t.truthy(hasErrorMsg)
hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/zero/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)

path_ = 'test/fixtures/test-data/files/zero-files/zero.txt'
result = await runcli('push', path_, args)
stdout = result.stdout.split('\n')
hasErrorMsg = stdout.find(item => item.includes('> You can not push empty files, please add some data and try again'))
t.truthy(hasErrorMsg)
hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/zero/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)

path_ = 'test/fixtures/test-data/files/zero-files/zero.json'
result = await runcli('push', path_, args)
stdout = result.stdout.split('\n')
hasErrorMsg = stdout.find(item => item.includes('> You can not push empty files, please add some data and try again'))
t.truthy(hasErrorMsg)
hasPublishedMessage = stdout.find(item => item.includes('your data is published!'))
hasURLtoShowcase = stdout.find(item => item.includes('https://datahub.io/test/zero/v/'))
t.truthy(hasPublishedMessage)
t.truthy(hasURLtoShowcase)

path_ = 'test/fixtures/test-data/files/zero-files/zero.xls'
result = await runcli('push', path_, args)
stdout = result.stdout.split('\n')
hasErrorMsg = stdout.find(item => item.includes('> You can not push empty files, please add some data and try again'))
let hasErrorMsg = stdout.find(item => item.includes('You can not push empty files, please add some data and try again'))
t.truthy(hasErrorMsg)
})

Expand Down

0 comments on commit b780ad2

Please sign in to comment.