박건희 / emon_bot

Go to a project
Toggle navigation Toggle navigation pinning
노영우
a5562189 2 parents 537fba16 f31e37b7

merge 'translator' into 'master'

Expand all
Inline Side-by-side
Showing 341 changed files with 63 additions and 4735 deletions
config.json
node_modules/
*.traineddata
......
MIT LicenseCopyright (c) 2022 gunhee park and other contributors
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
\ No newline at end of file
......@@ -2,16 +2,44 @@
## About The Project
* 각종 편의기능 통합 서비스를 제공하는 챗봇입니다.
* 도라에몽처럼 각종 편의기능을 통합적(All-in-One)으로 서비스할 수 있는 디스코드 챗봇입니다.
* 현재 제공하고 있는 기능으로는 아래 세 가지가 있습니다.
- 이미지 텍스트 인식 (OCR)
- 언어번역
- 필기구 가격비교
## Built With
* [Node.js](https://nodejs.org/ko/)
* [discord.js](https://discord.js.org)
* [puppeteer](https://pptr.dev/)
* [tesseract.js](https://tesseract.projectnaptha.com/)
* [translatte](https://www.npmjs.com/package/translatte)
## Getting Started
* [봇 초대 링크](https://discord.com/api/oauth2/authorize?client_id=974262592388337695&permissions=8&scope=bot)
* aws EC2 사용해서 배포 예정
* aws EC2 사용해서 배포
## Installation
1. Clone the repo
```
git clone http://khuhub.khu.ac.kr/2017103984/emon_bot.git
```
2. Install NPM packages
```
npm install
```
3. 디스코드 회원가입
4. [Discord Developer Portal](https://discord.com/developers/applications)에 들어가서 New Application을 생성
5. Application 생성 후 왼쪽에 보이는 사이드바에서 Bot을 눌러서 디스코드 봇 생성
6. 디스코드 봇 생성 후 토큰 복사 (외부 유출 주의)
![토큰 화면](https://blog.kakaocdn.net/dn/bO6T7F/btq9R5dSFbr/K3gS0pnjWEYiK1KwgeMqzk/img.png)
7. 다시 왼쪽 사이드바에서 OAuth2를 눌러 들어간 후 SCOPES에서 bot을 체크
8. 생성되는 주소로 이동해서 디스코드 봇을 본인의 서버로 초대
## Usage
......@@ -19,11 +47,19 @@
**!도움말** - 봇에 대한 명령어 리스트들을 불러옵니다.
**!도움말** <명령어> - <명령어>에 해당하는 명령어의 사용방법을 조회합니다.
**!ocr** <사용 언어> - 이미지를 첨부하고 <사용 언어>를 입력하면, 이미지에 있는 텍스트를 추출합니다.
**!문구** <채팅> - <채팅>에 대한 검색결과를 보여줍니다.
**!번역** <채팅> - <채팅>에 대한 번역결과를 보여줍니다.
**!나라** - 봇이 띄워주는 버튼을 눌러서 번역할 언어(나라)를 선택합니다.
**!문구** <채팅> - <채팅>에 대한 검색결과를 보여줍니다.
* 사용 예시 (추가 예정)
* 사용 예시
**도움말 명령어 실행화면**
![도움말_실행화면](/uploads/67c60458abcd7d9993f4b5c5fbefcba1/도움말_실행화면.png)
**ocr 명령어 실행화면**
![ocr_실행화면](/uploads/b2b5490a79eadad12d361c4ca9c62d2a/ocr_실행화면.png)
**번역, 나라 명령어 실행화면**
![번역_실행화면](/uploads/c802abe42bd24fbe45792ea44c2aa2c2/번역_실행화면.png)
**문구 명령어 실행화면**
![문구_실행화면](/uploads/4984a7457ff7bc10610a823567792a3f/문구_실행화면.png)
## Contributing
......
This file is too large to display.
This file is too large to display.
This file is too large to display.
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
case `uname` in
*CYGWIN*|*MINGW*|*MSYS*) basedir=`cygpath -w "$basedir"`;;
esac
if [ -x "$basedir/node" ]; then
exec "$basedir/node" "$basedir/../jpeg-autorotate/src/cli.js" "$@"
else
exec node "$basedir/../jpeg-autorotate/src/cli.js" "$@"
fi
@ECHO off
GOTO start
:find_dp0
SET dp0=%~dp0
EXIT /b
:start
SETLOCAL
CALL :find_dp0
IF EXIST "%dp0%\node.exe" (
SET "_prog=%dp0%\node.exe"
) ELSE (
SET "_prog=node"
SET PATHEXT=%PATHEXT:;.JS;=;%
)
endLocal & goto #_undefined_# 2>NUL || title %COMSPEC% & "%_prog%" "%dp0%\..\jpeg-autorotate\src\cli.js" %*
#!/usr/bin/env pwsh
$basedir=Split-Path $MyInvocation.MyCommand.Definition -Parent
$exe=""
if ($PSVersionTable.PSVersion -lt "6.0" -or $IsWindows) {
# Fix case when both the Windows and Linux builds of Node
# are installed in the same directory
$exe=".exe"
}
$ret=0
if (Test-Path "$basedir/node$exe") {
# Support pipeline input
if ($MyInvocation.ExpectingInput) {
$input | & "$basedir/node$exe" "$basedir/../jpeg-autorotate/src/cli.js" $args
} else {
& "$basedir/node$exe" "$basedir/../jpeg-autorotate/src/cli.js" $args
}
$ret=$LASTEXITCODE
} else {
# Support pipeline input
if ($MyInvocation.ExpectingInput) {
$input | & "node$exe" "$basedir/../jpeg-autorotate/src/cli.js" $args
} else {
& "node$exe" "$basedir/../jpeg-autorotate/src/cli.js" $args
}
$ret=$LASTEXITCODE
}
exit $ret
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
case `uname` in
*CYGWIN*|*MINGW*|*MSYS*) basedir=`cygpath -w "$basedir"`;;
esac
if [ -x "$basedir/node" ]; then
exec "$basedir/node" "$basedir/../opencollective-postinstall/index.js" "$@"
else
exec node "$basedir/../opencollective-postinstall/index.js" "$@"
fi
@ECHO off
GOTO start
:find_dp0
SET dp0=%~dp0
EXIT /b
:start
SETLOCAL
CALL :find_dp0
IF EXIST "%dp0%\node.exe" (
SET "_prog=%dp0%\node.exe"
) ELSE (
SET "_prog=node"
SET PATHEXT=%PATHEXT:;.JS;=;%
)
endLocal & goto #_undefined_# 2>NUL || title %COMSPEC% & "%_prog%" "%dp0%\..\opencollective-postinstall\index.js" %*
#!/usr/bin/env pwsh
$basedir=Split-Path $MyInvocation.MyCommand.Definition -Parent
$exe=""
if ($PSVersionTable.PSVersion -lt "6.0" -or $IsWindows) {
# Fix case when both the Windows and Linux builds of Node
# are installed in the same directory
$exe=".exe"
}
$ret=0
if (Test-Path "$basedir/node$exe") {
# Support pipeline input
if ($MyInvocation.ExpectingInput) {
$input | & "$basedir/node$exe" "$basedir/../opencollective-postinstall/index.js" $args
} else {
& "$basedir/node$exe" "$basedir/../opencollective-postinstall/index.js" $args
}
$ret=$LASTEXITCODE
} else {
# Support pipeline input
if ($MyInvocation.ExpectingInput) {
$input | & "node$exe" "$basedir/../opencollective-postinstall/index.js" $args
} else {
& "node$exe" "$basedir/../opencollective-postinstall/index.js" $args
}
$ret=$LASTEXITCODE
}
exit $ret
This diff is collapsed. Click to expand it.
MIT License
Copyright (c) 2018 Szymon Marczak
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
# http-timer
> Timings for HTTP requests
[![Build Status](https://travis-ci.org/szmarczak/http-timer.svg?branch=master)](https://travis-ci.org/szmarczak/http-timer)
[![Coverage Status](https://coveralls.io/repos/github/szmarczak/http-timer/badge.svg?branch=master)](https://coveralls.io/github/szmarczak/http-timer?branch=master)
[![install size](https://packagephobia.now.sh/badge?p=@szmarczak/http-timer)](https://packagephobia.now.sh/result?p=@szmarczak/http-timer)
Inspired by the [`request` package](https://github.com/request/request).
## Usage
```js
'use strict';
const https = require('https');
const timer = require('@szmarczak/http-timer');
const request = https.get('https://httpbin.org/anything');
const timings = timer(request);
request.on('response', response => {
response.on('data', () => {}); // Consume the data somehow
response.on('end', () => {
console.log(timings);
});
});
// { start: 1535708511443,
// socket: 1535708511444,
// lookup: 1535708511444,
// connect: 1535708511582,
// upload: 1535708511887,
// response: 1535708512037,
// end: 1535708512040,
// phases:
// { wait: 1,
// dns: 0,
// tcp: 138,
// request: 305,
// firstByte: 150,
// download: 3,
// total: 597 } }
```
## API
### timer(request)
Returns: `Object`
- `start` - Time when the request started.
- `socket` - Time when a socket was assigned to the request.
- `lookup` - Time when the DNS lookup finished.
- `connect` - Time when the socket successfully connected.
- `upload` - Time when the request finished uploading.
- `response` - Time when the request fired the `response` event.
- `end` - Time when the response fired the `end` event.
- `error` - Time when the request fired the `error` event.
- `phases`
- `wait` - `timings.socket - timings.start`
- `dns` - `timings.lookup - timings.socket`
- `tcp` - `timings.connect - timings.lookup`
- `request` - `timings.upload - timings.connect`
- `firstByte` - `timings.response - timings.upload`
- `download` - `timings.end - timings.response`
- `total` - `timings.end - timings.start` or `timings.error - timings.start`
**Note**: The time is a `number` representing the milliseconds elapsed since the UNIX epoch.
## License
MIT
{
"name": "@szmarczak/http-timer",
"version": "1.1.2",
"description": "Timings for HTTP requests",
"main": "source",
"engines": {
"node": ">=6"
},
"scripts": {
"test": "xo && nyc ava",
"coveralls": "nyc report --reporter=text-lcov | coveralls"
},
"files": [
"source"
],
"keywords": [
"http",
"https",
"timer",
"timings"
],
"repository": {
"type": "git",
"url": "git+https://github.com/szmarczak/http-timer.git"
},
"author": "Szymon Marczak",
"license": "MIT",
"bugs": {
"url": "https://github.com/szmarczak/http-timer/issues"
},
"homepage": "https://github.com/szmarczak/http-timer#readme",
"xo": {
"rules": {
"unicorn/filename-case": "camelCase"
}
},
"devDependencies": {
"ava": "^0.25.0",
"coveralls": "^3.0.2",
"p-event": "^2.1.0",
"nyc": "^12.0.2",
"xo": "^0.22.0"
},
"dependencies": {
"defer-to-connect": "^1.0.1"
}
}
'use strict';
const deferToConnect = require('defer-to-connect');
module.exports = request => {
const timings = {
start: Date.now(),
socket: null,
lookup: null,
connect: null,
upload: null,
response: null,
end: null,
error: null,
phases: {
wait: null,
dns: null,
tcp: null,
request: null,
firstByte: null,
download: null,
total: null
}
};
const handleError = origin => {
const emit = origin.emit.bind(origin);
origin.emit = (event, ...args) => {
// Catches the `error` event
if (event === 'error') {
timings.error = Date.now();
timings.phases.total = timings.error - timings.start;
origin.emit = emit;
}
// Saves the original behavior
return emit(event, ...args);
};
};
let uploadFinished = false;
const onUpload = () => {
timings.upload = Date.now();
timings.phases.request = timings.upload - timings.connect;
};
handleError(request);
request.once('socket', socket => {
timings.socket = Date.now();
timings.phases.wait = timings.socket - timings.start;
const lookupListener = () => {
timings.lookup = Date.now();
timings.phases.dns = timings.lookup - timings.socket;
};
socket.once('lookup', lookupListener);
deferToConnect(socket, () => {
timings.connect = Date.now();
if (timings.lookup === null) {
socket.removeListener('lookup', lookupListener);
timings.lookup = timings.connect;
timings.phases.dns = timings.lookup - timings.socket;
}
timings.phases.tcp = timings.connect - timings.lookup;
if (uploadFinished && !timings.upload) {
onUpload();
}
});
});
request.once('finish', () => {
uploadFinished = true;
if (timings.connect) {
onUpload();
}
});
request.once('response', response => {
timings.response = Date.now();
timings.phases.firstByte = timings.response - timings.upload;
handleError(response);
response.once('end', () => {
timings.end = Date.now();
timings.phases.download = timings.end - timings.response;
timings.phases.total = timings.end - timings.start;
});
});
return timings;
};
language: node_js
node_js:
- "0.10"
- "0.12"
- "iojs"
Copyright (c) 2010-2014 Caolan McMahon
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
This diff is collapsed. Click to expand it.
{
"name": "async",
"description": "Higher-order functions and common patterns for asynchronous code",
"version": "0.9.2",
"main": "lib/async.js",
"keywords": [
"async",
"callback",
"utility",
"module"
],
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/caolan/async.git"
},
"devDependencies": {
"nodeunit": ">0.0.0",
"uglify-js": "1.2.x",
"nodelint": ">0.0.0",
"lodash": ">=2.4.1"
},
"moduleType": [
"amd",
"globals",
"node"
],
"ignore": [
"**/.*",
"node_modules",
"bower_components",
"test",
"tests"
],
"authors": [
"Caolan McMahon"
]
}
\ No newline at end of file
{
"name": "async",
"description": "Higher-order functions and common patterns for asynchronous code",
"version": "0.9.2",
"keywords": [
"async",
"callback",
"utility",
"module"
],
"license": "MIT",
"repository": "caolan/async",
"scripts": [
"lib/async.js"
]
}
\ No newline at end of file
This diff is collapsed. Click to expand it.
{
"name": "async",
"description": "Higher-order functions and common patterns for asynchronous code",
"main": "lib/async.js",
"author": "Caolan McMahon",
"version": "0.9.2",
"keywords": [
"async",
"callback",
"utility",
"module"
],
"repository": {
"type": "git",
"url": "https://github.com/caolan/async.git"
},
"bugs": {
"url": "https://github.com/caolan/async/issues"
},
"license": "MIT",
"devDependencies": {
"nodeunit": ">0.0.0",
"uglify-js": "1.2.x",
"nodelint": ">0.0.0",
"lodash": ">=2.4.1"
},
"jam": {
"main": "lib/async.js",
"include": [
"lib/async.js",
"README.md",
"LICENSE"
],
"categories": [
"Utilities"
]
},
"scripts": {
"test": "nodeunit test/test-async.js"
},
"spm": {
"main": "lib/async.js"
},
"volo": {
"main": "lib/async.js",
"ignore": [
"**/.*",
"node_modules",
"bower_components",
"test",
"tests"
]
}
}
\ No newline at end of file
#!/usr/bin/env node
// This should probably be its own module but complaints about bower/etc.
// support keep coming up and I'd rather just enable the workflow here for now
// and figure out where this should live later. -- @beaugunderson
var fs = require('fs');
var _ = require('lodash');
var packageJson = require('../package.json');
var IGNORES = ['**/.*', 'node_modules', 'bower_components', 'test', 'tests'];
var INCLUDES = ['lib/async.js', 'README.md', 'LICENSE'];
var REPOSITORY_NAME = 'caolan/async';
packageJson.jam = {
main: packageJson.main,
include: INCLUDES,
categories: ['Utilities']
};
packageJson.spm = {
main: packageJson.main
};
packageJson.volo = {
main: packageJson.main,
ignore: IGNORES
};
var bowerSpecific = {
moduleType: ['amd', 'globals', 'node'],
ignore: IGNORES,
authors: [packageJson.author]
};
var bowerInclude = ['name', 'description', 'version', 'main', 'keywords',
'license', 'homepage', 'repository', 'devDependencies'];
var componentSpecific = {
repository: REPOSITORY_NAME,
scripts: [packageJson.main]
};
var componentInclude = ['name', 'description', 'version', 'keywords',
'license'];
var bowerJson = _.merge({}, _.pick(packageJson, bowerInclude), bowerSpecific);
var componentJson = _.merge({}, _.pick(packageJson, componentInclude), componentSpecific);
fs.writeFileSync('./bower.json', JSON.stringify(bowerJson, null, 2));
fs.writeFileSync('./component.json', JSON.stringify(componentJson, null, 2));
fs.writeFileSync('./package.json', JSON.stringify(packageJson, null, 2));
MIT License
Copyright (c) 2017 Luke Childs
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
# cacheable-request
> Wrap native HTTP requests with RFC compliant cache support
[![Build Status](https://travis-ci.org/lukechilds/cacheable-request.svg?branch=master)](https://travis-ci.org/lukechilds/cacheable-request)
[![Coverage Status](https://coveralls.io/repos/github/lukechilds/cacheable-request/badge.svg?branch=master)](https://coveralls.io/github/lukechilds/cacheable-request?branch=master)
[![npm](https://img.shields.io/npm/dm/cacheable-request.svg)](https://www.npmjs.com/package/cacheable-request)
[![npm](https://img.shields.io/npm/v/cacheable-request.svg)](https://www.npmjs.com/package/cacheable-request)
[RFC 7234](http://httpwg.org/specs/rfc7234.html) compliant HTTP caching for native Node.js HTTP/HTTPS requests. Caching works out of the box in memory or is easily pluggable with a wide range of storage adapters.
**Note:** This is a low level wrapper around the core HTTP modules, it's not a high level request library.
## Features
- Only stores cacheable responses as defined by RFC 7234
- Fresh cache entries are served directly from cache
- Stale cache entries are revalidated with `If-None-Match`/`If-Modified-Since` headers
- 304 responses from revalidation requests use cached body
- Updates `Age` header on cached responses
- Can completely bypass cache on a per request basis
- In memory cache by default
- Official support for Redis, MongoDB, SQLite, PostgreSQL and MySQL storage adapters
- Easily plug in your own or third-party storage adapters
- If DB connection fails, cache is automatically bypassed ([disabled by default](#optsautomaticfailover))
- Adds cache support to any existing HTTP code with minimal changes
- Uses [http-cache-semantics](https://github.com/pornel/http-cache-semantics) internally for HTTP RFC 7234 compliance
## Install
```shell
npm install cacheable-request
```
## Usage
```js
const http = require('http');
const CacheableRequest = require('cacheable-request');
// Then instead of
const req = http.request('http://example.com', cb);
req.end();
// You can do
const cacheableRequest = new CacheableRequest(http.request);
const cacheReq = cacheableRequest('http://example.com', cb);
cacheReq.on('request', req => req.end());
// Future requests to 'example.com' will be returned from cache if still valid
// You pass in any other http.request API compatible method to be wrapped with cache support:
const cacheableRequest = new CacheableRequest(https.request);
const cacheableRequest = new CacheableRequest(electron.net);
```
## Storage Adapters
`cacheable-request` uses [Keyv](https://github.com/lukechilds/keyv) to support a wide range of storage adapters.
For example, to use Redis as a cache backend, you just need to install the official Redis Keyv storage adapter:
```
npm install @keyv/redis
```
And then you can pass `CacheableRequest` your connection string:
```js
const cacheableRequest = new CacheableRequest(http.request, 'redis://user:pass@localhost:6379');
```
[View all official Keyv storage adapters.](https://github.com/lukechilds/keyv#official-storage-adapters)
Keyv also supports anything that follows the Map API so it's easy to write your own storage adapter or use a third-party solution.
e.g The following are all valid storage adapters
```js
const storageAdapter = new Map();
// or
const storageAdapter = require('./my-storage-adapter');
// or
const QuickLRU = require('quick-lru');
const storageAdapter = new QuickLRU({ maxSize: 1000 });
const cacheableRequest = new CacheableRequest(http.request, storageAdapter);
```
View the [Keyv docs](https://github.com/lukechilds/keyv) for more information on how to use storage adapters.
## API
### new cacheableRequest(request, [storageAdapter])
Returns the provided request function wrapped with cache support.
#### request
Type: `function`
Request function to wrap with cache support. Should be [`http.request`](https://nodejs.org/api/http.html#http_http_request_options_callback) or a similar API compatible request function.
#### storageAdapter
Type: `Keyv storage adapter`<br>
Default: `new Map()`
A [Keyv](https://github.com/lukechilds/keyv) storage adapter instance, or connection string if using with an official Keyv storage adapter.
### Instance
#### cacheableRequest(opts, [cb])
Returns an event emitter.
##### opts
Type: `object`, `string`
- Any of the default request functions options.
- Any [`http-cache-semantics`](https://github.com/kornelski/http-cache-semantics#constructor-options) options.
- Any of the following:
###### opts.cache
Type: `boolean`<br>
Default: `true`
If the cache should be used. Setting this to false will completely bypass the cache for the current request.
###### opts.strictTtl
Type: `boolean`<br>
Default: `false`
If set to `true` once a cached resource has expired it is deleted and will have to be re-requested.
If set to `false` (default), after a cached resource's TTL expires it is kept in the cache and will be revalidated on the next request with `If-None-Match`/`If-Modified-Since` headers.
###### opts.maxTtl
Type: `number`<br>
Default: `undefined`
Limits TTL. The `number` represents milliseconds.
###### opts.automaticFailover
Type: `boolean`<br>
Default: `false`
When set to `true`, if the DB connection fails we will automatically fallback to a network request. DB errors will still be emitted to notify you of the problem even though the request callback may succeed.
###### opts.forceRefresh
Type: `boolean`<br>
Default: `false`
Forces refreshing the cache. If the response could be retrieved from the cache, it will perform a new request and override the cache instead.
##### cb
Type: `function`
The callback function which will receive the response as an argument.
The response can be either a [Node.js HTTP response stream](https://nodejs.org/api/http.html#http_class_http_incomingmessage) or a [responselike object](https://github.com/lukechilds/responselike). The response will also have a `fromCache` property set with a boolean value.
##### .on('request', request)
`request` event to get the request object of the request.
**Note:** This event will only fire if an HTTP request is actually made, not when a response is retrieved from cache. However, you should always handle the `request` event to end the request and handle any potential request errors.
##### .on('response', response)
`response` event to get the response object from the HTTP request or cache.
##### .on('error', error)
`error` event emitted in case of an error with the cache.
Errors emitted here will be an instance of `CacheableRequest.RequestError` or `CacheableRequest.CacheError`. You will only ever receive a `RequestError` if the request function throws (normally caused by invalid user input). Normal request errors should be handled inside the `request` event.
To properly handle all error scenarios you should use the following pattern:
```js
cacheableRequest('example.com', cb)
.on('error', err => {
if (err instanceof CacheableRequest.CacheError) {
handleCacheError(err); // Cache error
} else if (err instanceof CacheableRequest.RequestError) {
handleRequestError(err); // Request function thrown
}
})
.on('request', req => {
req.on('error', handleRequestError); // Request error emitted
req.end();
});
```
**Note:** Database connection errors are emitted here, however `cacheable-request` will attempt to re-request the resource and bypass the cache on a connection error. Therefore a database connection error doesn't necessarily mean the request won't be fulfilled.
## License
MIT © Luke Childs
'use strict';
const {PassThrough: PassThroughStream} = require('stream');
module.exports = options => {
options = {...options};
const {array} = options;
let {encoding} = options;
const isBuffer = encoding === 'buffer';
let objectMode = false;
if (array) {
objectMode = !(encoding || isBuffer);
} else {
encoding = encoding || 'utf8';
}
if (isBuffer) {
encoding = null;
}
const stream = new PassThroughStream({objectMode});
if (encoding) {
stream.setEncoding(encoding);
}
let length = 0;
const chunks = [];
stream.on('data', chunk => {
chunks.push(chunk);
if (objectMode) {
length = chunks.length;
} else {
length += chunk.length;
}
});
stream.getBufferedValue = () => {
if (array) {
return chunks;
}
return isBuffer ? Buffer.concat(chunks, length) : chunks.join('');
};
stream.getBufferedLength = () => length;
return stream;
};
/// <reference types="node"/>
import {Stream} from 'stream';
declare class MaxBufferErrorClass extends Error {
readonly name: 'MaxBufferError';
constructor();
}
declare namespace getStream {
interface Options {
/**
Maximum length of the returned string. If it exceeds this value before the stream ends, the promise will be rejected with a `MaxBufferError` error.
@default Infinity
*/
readonly maxBuffer?: number;
}
interface OptionsWithEncoding<EncodingType = BufferEncoding> extends Options {
/**
[Encoding](https://nodejs.org/api/buffer.html#buffer_buffer) of the incoming stream.
@default 'utf8'
*/
readonly encoding?: EncodingType;
}
type MaxBufferError = MaxBufferErrorClass;
}
declare const getStream: {
/**
Get the `stream` as a string.
@returns A promise that resolves when the end event fires on the stream, indicating that there is no more data to be read. The stream is switched to flowing mode.
@example
```
import * as fs from 'fs';
import getStream = require('get-stream');
(async () => {
const stream = fs.createReadStream('unicorn.txt');
console.log(await getStream(stream));
// ,,))))))));,
// __)))))))))))))),
// \|/ -\(((((''''((((((((.
// -*-==//////(('' . `)))))),
// /|\ ))| o ;-. '((((( ,(,
// ( `| / ) ;))))' ,_))^;(~
// | | | ,))((((_ _____------~~~-. %,;(;(>';'~
// o_); ; )))(((` ~---~ `:: \ %%~~)(v;(`('~
// ; ''''```` `: `:::|\,__,%% );`'; ~
// | _ ) / `:|`----' `-'
// ______/\/~ | / /
// /~;;.____/;;' / ___--,-( `;;;/
// / // _;______;'------~~~~~ /;;/\ /
// // | | / ; \;;,\
// (<_ | ; /',/-----' _>
// \_| ||_ //~;~~~~~~~~~
// `\_| (,~~
// \~\
// ~~
})();
```
*/
(stream: Stream, options?: getStream.OptionsWithEncoding): Promise<string>;
/**
Get the `stream` as a buffer.
It honors the `maxBuffer` option as above, but it refers to byte length rather than string length.
*/
buffer(
stream: Stream,
options?: getStream.OptionsWithEncoding
): Promise<Buffer>;
/**
Get the `stream` as an array of values.
It honors both the `maxBuffer` and `encoding` options. The behavior changes slightly based on the encoding chosen:
- When `encoding` is unset, it assumes an [object mode stream](https://nodesource.com/blog/understanding-object-streams/) and collects values emitted from `stream` unmodified. In this case `maxBuffer` refers to the number of items in the array (not the sum of their sizes).
- When `encoding` is set to `buffer`, it collects an array of buffers. `maxBuffer` refers to the summed byte lengths of every buffer in the array.
- When `encoding` is set to anything else, it collects an array of strings. `maxBuffer` refers to the summed character lengths of every string in the array.
*/
array<StreamObjectModeType>(
stream: Stream,
options?: getStream.Options
): Promise<StreamObjectModeType[]>;
array(
stream: Stream,
options: getStream.OptionsWithEncoding<'buffer'>
): Promise<Buffer[]>;
array(
stream: Stream,
options: getStream.OptionsWithEncoding<BufferEncoding>
): Promise<string[]>;
MaxBufferError: typeof MaxBufferErrorClass;
// TODO: Remove this for the next major release
default: typeof getStream;
};
export = getStream;
'use strict';
const {constants: BufferConstants} = require('buffer');
const pump = require('pump');
const bufferStream = require('./buffer-stream');
class MaxBufferError extends Error {
constructor() {
super('maxBuffer exceeded');
this.name = 'MaxBufferError';
}
}
async function getStream(inputStream, options) {
if (!inputStream) {
return Promise.reject(new Error('Expected a stream'));
}
options = {
maxBuffer: Infinity,
...options
};
const {maxBuffer} = options;
let stream;
await new Promise((resolve, reject) => {
const rejectPromise = error => {
// Don't retrieve an oversized buffer.
if (error && stream.getBufferedLength() <= BufferConstants.MAX_LENGTH) {
error.bufferedData = stream.getBufferedValue();
}
reject(error);
};
stream = pump(inputStream, bufferStream(options), error => {
if (error) {
rejectPromise(error);
return;
}
resolve();
});
stream.on('data', () => {
if (stream.getBufferedLength() > maxBuffer) {
rejectPromise(new MaxBufferError());
}
});
});
return stream.getBufferedValue();
}
module.exports = getStream;
// TODO: Remove this for the next major release
module.exports.default = getStream;
module.exports.buffer = (stream, options) => getStream(stream, {...options, encoding: 'buffer'});
module.exports.array = (stream, options) => getStream(stream, {...options, array: true});
module.exports.MaxBufferError = MaxBufferError;
MIT License
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (https://sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
{
"name": "get-stream",
"version": "5.2.0",
"description": "Get a stream as a string, buffer, or array",
"license": "MIT",
"repository": "sindresorhus/get-stream",
"funding": "https://github.com/sponsors/sindresorhus",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "https://sindresorhus.com"
},
"engines": {
"node": ">=8"
},
"scripts": {
"test": "xo && ava && tsd"
},
"files": [
"index.js",
"index.d.ts",
"buffer-stream.js"
],
"keywords": [
"get",
"stream",
"promise",
"concat",
"string",
"text",
"buffer",
"read",
"data",
"consume",
"readable",
"readablestream",
"array",
"object"
],
"dependencies": {
"pump": "^3.0.0"
},
"devDependencies": {
"@types/node": "^12.0.7",
"ava": "^2.0.0",
"into-stream": "^5.0.0",
"tsd": "^0.7.2",
"xo": "^0.24.0"
}
}
# get-stream [![Build Status](https://travis-ci.com/sindresorhus/get-stream.svg?branch=master)](https://travis-ci.com/github/sindresorhus/get-stream)
> Get a stream as a string, buffer, or array
## Install
```
$ npm install get-stream
```
## Usage
```js
const fs = require('fs');
const getStream = require('get-stream');
(async () => {
const stream = fs.createReadStream('unicorn.txt');
console.log(await getStream(stream));
/*
,,))))))));,
__)))))))))))))),
\|/ -\(((((''''((((((((.
-*-==//////(('' . `)))))),
/|\ ))| o ;-. '((((( ,(,
( `| / ) ;))))' ,_))^;(~
| | | ,))((((_ _____------~~~-. %,;(;(>';'~
o_); ; )))(((` ~---~ `:: \ %%~~)(v;(`('~
; ''''```` `: `:::|\,__,%% );`'; ~
| _ ) / `:|`----' `-'
______/\/~ | / /
/~;;.____/;;' / ___--,-( `;;;/
/ // _;______;'------~~~~~ /;;/\ /
// | | / ; \;;,\
(<_ | ; /',/-----' _>
\_| ||_ //~;~~~~~~~~~
`\_| (,~~
\~\
~~
*/
})();
```
## API
The methods returns a promise that resolves when the `end` event fires on the stream, indicating that there is no more data to be read. The stream is switched to flowing mode.
### getStream(stream, options?)
Get the `stream` as a string.
#### options
Type: `object`
##### encoding
Type: `string`\
Default: `'utf8'`
[Encoding](https://nodejs.org/api/buffer.html#buffer_buffer) of the incoming stream.
##### maxBuffer
Type: `number`\
Default: `Infinity`
Maximum length of the returned string. If it exceeds this value before the stream ends, the promise will be rejected with a `getStream.MaxBufferError` error.
### getStream.buffer(stream, options?)
Get the `stream` as a buffer.
It honors the `maxBuffer` option as above, but it refers to byte length rather than string length.
### getStream.array(stream, options?)
Get the `stream` as an array of values.
It honors both the `maxBuffer` and `encoding` options. The behavior changes slightly based on the encoding chosen:
- When `encoding` is unset, it assumes an [object mode stream](https://nodesource.com/blog/understanding-object-streams/) and collects values emitted from `stream` unmodified. In this case `maxBuffer` refers to the number of items in the array (not the sum of their sizes).
- When `encoding` is set to `buffer`, it collects an array of buffers. `maxBuffer` refers to the summed byte lengths of every buffer in the array.
- When `encoding` is set to anything else, it collects an array of strings. `maxBuffer` refers to the summed character lengths of every string in the array.
## Errors
If the input stream emits an `error` event, the promise will be rejected with the error. The buffered data will be attached to the `bufferedData` property of the error.
```js
(async () => {
try {
await getStream(streamThatErrorsAtTheEnd('unicorn'));
} catch (error) {
console.log(error.bufferedData);
//=> 'unicorn'
}
})()
```
## FAQ
### How is this different from [`concat-stream`](https://github.com/maxogden/concat-stream)?
This module accepts a stream instead of being one and returns a promise instead of using a callback. The API is simpler and it only supports returning a string, buffer, or array. It doesn't have a fragile type inference. You explicitly choose what you want. And it doesn't depend on the huge `readable-stream` package.
## Related
- [get-stdin](https://github.com/sindresorhus/get-stdin) - Get stdin as a string or buffer
---
<div align="center">
<b>
<a href="https://tidelift.com/subscription/pkg/npm-get-stream?utm_source=npm-get-stream&utm_medium=referral&utm_campaign=readme">Get professional support for this package with a Tidelift subscription</a>
</b>
<br>
<sub>
Tidelift helps make open source sustainable for maintainers while giving companies<br>assurances about security, maintenance, and licensing for their dependencies.
</sub>
</div>
/**
Lowercase the keys of an object.
@returns A new object with the keys lowercased.
@example
```
import lowercaseKeys = require('lowercase-keys');
lowercaseKeys({FOO: true, bAr: false});
//=> {foo: true, bar: false}
```
*/
declare function lowercaseKeys<T extends unknown>(object: {[key: string]: T}): {[key: string]: T};
export = lowercaseKeys;
'use strict';
module.exports = object => {
const result = {};
for (const [key, value] of Object.entries(object)) {
result[key.toLowerCase()] = value;
}
return result;
};
MIT License
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
{
"name": "lowercase-keys",
"version": "2.0.0",
"description": "Lowercase the keys of an object",
"license": "MIT",
"repository": "sindresorhus/lowercase-keys",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"engines": {
"node": ">=8"
},
"scripts": {
"test": "xo && ava && tsd"
},
"files": [
"index.js",
"index.d.ts"
],
"keywords": [
"object",
"assign",
"extend",
"properties",
"lowercase",
"lower-case",
"case",
"keys",
"key"
],
"devDependencies": {
"ava": "^1.4.1",
"tsd": "^0.7.2",
"xo": "^0.24.0"
}
}
# lowercase-keys [![Build Status](https://travis-ci.org/sindresorhus/lowercase-keys.svg?branch=master)](https://travis-ci.org/sindresorhus/lowercase-keys)
> Lowercase the keys of an object
## Install
```
$ npm install lowercase-keys
```
## Usage
```js
const lowercaseKeys = require('lowercase-keys');
lowercaseKeys({FOO: true, bAr: false});
//=> {foo: true, bar: false}
```
## API
### lowercaseKeys(object)
Returns a new object with the keys lowercased.
## License
MIT © [Sindre Sorhus](https://sindresorhus.com)
{
"name": "cacheable-request",
"version": "6.1.0",
"description": "Wrap native HTTP requests with RFC compliant cache support",
"license": "MIT",
"repository": "lukechilds/cacheable-request",
"author": "Luke Childs <lukechilds123@gmail.com> (http://lukechilds.co.uk)",
"main": "src/index.js",
"engines": {
"node": ">=8"
},
"scripts": {
"test": "xo && nyc ava",
"coverage": "nyc report --reporter=text-lcov | coveralls"
},
"files": [
"src"
],
"keywords": [
"HTTP",
"HTTPS",
"cache",
"caching",
"layer",
"cacheable",
"RFC 7234",
"RFC",
"7234",
"compliant"
],
"dependencies": {
"clone-response": "^1.0.2",
"get-stream": "^5.1.0",
"http-cache-semantics": "^4.0.0",
"keyv": "^3.0.0",
"lowercase-keys": "^2.0.0",
"normalize-url": "^4.1.0",
"responselike": "^1.0.2"
},
"devDependencies": {
"@keyv/sqlite": "^2.0.0",
"ava": "^1.1.0",
"coveralls": "^3.0.0",
"create-test-server": "3.0.0",
"delay": "^4.0.0",
"eslint-config-xo-lukechilds": "^1.0.0",
"nyc": "^14.1.1",
"pify": "^4.0.0",
"sqlite3": "^4.0.2",
"this": "^1.0.2",
"xo": "^0.23.0"
},
"xo": {
"extends": "xo-lukechilds"
}
}
'use strict';
const EventEmitter = require('events');
const urlLib = require('url');
const normalizeUrl = require('normalize-url');
const getStream = require('get-stream');
const CachePolicy = require('http-cache-semantics');
const Response = require('responselike');
const lowercaseKeys = require('lowercase-keys');
const cloneResponse = require('clone-response');
const Keyv = require('keyv');
class CacheableRequest {
constructor(request, cacheAdapter) {
if (typeof request !== 'function') {
throw new TypeError('Parameter `request` must be a function');
}
this.cache = new Keyv({
uri: typeof cacheAdapter === 'string' && cacheAdapter,
store: typeof cacheAdapter !== 'string' && cacheAdapter,
namespace: 'cacheable-request'
});
return this.createCacheableRequest(request);
}
createCacheableRequest(request) {
return (opts, cb) => {
let url;
if (typeof opts === 'string') {
url = normalizeUrlObject(urlLib.parse(opts));
opts = {};
} else if (opts instanceof urlLib.URL) {
url = normalizeUrlObject(urlLib.parse(opts.toString()));
opts = {};
} else {
const [pathname, ...searchParts] = (opts.path || '').split('?');
const search = searchParts.length > 0 ?
`?${searchParts.join('?')}` :
'';
url = normalizeUrlObject({ ...opts, pathname, search });
}
opts = {
headers: {},
method: 'GET',
cache: true,
strictTtl: false,
automaticFailover: false,
...opts,
...urlObjectToRequestOptions(url)
};
opts.headers = lowercaseKeys(opts.headers);
const ee = new EventEmitter();
const normalizedUrlString = normalizeUrl(
urlLib.format(url),
{
stripWWW: false,
removeTrailingSlash: false,
stripAuthentication: false
}
);
const key = `${opts.method}:${normalizedUrlString}`;
let revalidate = false;
let madeRequest = false;
const makeRequest = opts => {
madeRequest = true;
let requestErrored = false;
let requestErrorCallback;
const requestErrorPromise = new Promise(resolve => {
requestErrorCallback = () => {
if (!requestErrored) {
requestErrored = true;
resolve();
}
};
});
const handler = response => {
if (revalidate && !opts.forceRefresh) {
response.status = response.statusCode;
const revalidatedPolicy = CachePolicy.fromObject(revalidate.cachePolicy).revalidatedPolicy(opts, response);
if (!revalidatedPolicy.modified) {
const headers = revalidatedPolicy.policy.responseHeaders();
response = new Response(revalidate.statusCode, headers, revalidate.body, revalidate.url);
response.cachePolicy = revalidatedPolicy.policy;
response.fromCache = true;
}
}
if (!response.fromCache) {
response.cachePolicy = new CachePolicy(opts, response, opts);
response.fromCache = false;
}
let clonedResponse;
if (opts.cache && response.cachePolicy.storable()) {
clonedResponse = cloneResponse(response);
(async () => {
try {
const bodyPromise = getStream.buffer(response);
await Promise.race([
requestErrorPromise,
new Promise(resolve => response.once('end', resolve))
]);
if (requestErrored) {
return;
}
const body = await bodyPromise;
const value = {
cachePolicy: response.cachePolicy.toObject(),
url: response.url,
statusCode: response.fromCache ? revalidate.statusCode : response.statusCode,
body
};
let ttl = opts.strictTtl ? response.cachePolicy.timeToLive() : undefined;
if (opts.maxTtl) {
ttl = ttl ? Math.min(ttl, opts.maxTtl) : opts.maxTtl;
}
await this.cache.set(key, value, ttl);
} catch (error) {
ee.emit('error', new CacheableRequest.CacheError(error));
}
})();
} else if (opts.cache && revalidate) {
(async () => {
try {
await this.cache.delete(key);
} catch (error) {
ee.emit('error', new CacheableRequest.CacheError(error));
}
})();
}
ee.emit('response', clonedResponse || response);
if (typeof cb === 'function') {
cb(clonedResponse || response);
}
};
try {
const req = request(opts, handler);
req.once('error', requestErrorCallback);
req.once('abort', requestErrorCallback);
ee.emit('request', req);
} catch (error) {
ee.emit('error', new CacheableRequest.RequestError(error));
}
};
(async () => {
const get = async opts => {
await Promise.resolve();
const cacheEntry = opts.cache ? await this.cache.get(key) : undefined;
if (typeof cacheEntry === 'undefined') {
return makeRequest(opts);
}
const policy = CachePolicy.fromObject(cacheEntry.cachePolicy);
if (policy.satisfiesWithoutRevalidation(opts) && !opts.forceRefresh) {
const headers = policy.responseHeaders();
const response = new Response(cacheEntry.statusCode, headers, cacheEntry.body, cacheEntry.url);
response.cachePolicy = policy;
response.fromCache = true;
ee.emit('response', response);
if (typeof cb === 'function') {
cb(response);
}
} else {
revalidate = cacheEntry;
opts.headers = policy.revalidationHeaders(opts);
makeRequest(opts);
}
};
const errorHandler = error => ee.emit('error', new CacheableRequest.CacheError(error));
this.cache.once('error', errorHandler);
ee.on('response', () => this.cache.removeListener('error', errorHandler));
try {
await get(opts);
} catch (error) {
if (opts.automaticFailover && !madeRequest) {
makeRequest(opts);
}
ee.emit('error', new CacheableRequest.CacheError(error));
}
})();
return ee;
};
}
}
function urlObjectToRequestOptions(url) {
const options = { ...url };
options.path = `${url.pathname || '/'}${url.search || ''}`;
delete options.pathname;
delete options.search;
return options;
}
function normalizeUrlObject(url) {
// If url was parsed by url.parse or new URL:
// - hostname will be set
// - host will be hostname[:port]
// - port will be set if it was explicit in the parsed string
// Otherwise, url was from request options:
// - hostname or host may be set
// - host shall not have port encoded
return {
protocol: url.protocol,
auth: url.auth,
hostname: url.hostname || url.host || 'localhost',
port: url.port,
pathname: url.pathname,
search: url.search
};
}
CacheableRequest.RequestError = class extends Error {
constructor(error) {
super(error.message);
this.name = 'RequestError';
Object.assign(this, error);
}
};
CacheableRequest.CacheError = class extends Error {
constructor(error) {
super(error.message);
this.name = 'CacheError';
Object.assign(this, error);
}
};
module.exports = CacheableRequest;
MIT License
Copyright (c) 2017 Luke Childs
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
# clone-response
> Clone a Node.js HTTP response stream
[![Build Status](https://travis-ci.org/lukechilds/clone-response.svg?branch=master)](https://travis-ci.org/lukechilds/clone-response)
[![Coverage Status](https://coveralls.io/repos/github/lukechilds/clone-response/badge.svg?branch=master)](https://coveralls.io/github/lukechilds/clone-response?branch=master)
[![npm](https://img.shields.io/npm/dm/clone-response.svg)](https://www.npmjs.com/package/clone-response)
[![npm](https://img.shields.io/npm/v/clone-response.svg)](https://www.npmjs.com/package/clone-response)
Returns a new stream and copies over all properties and methods from the original response giving you a complete duplicate.
This is useful in situations where you need to consume the response stream but also want to pass an unconsumed stream somewhere else to be consumed later.
## Install
```shell
npm install --save clone-response
```
## Usage
```js
const http = require('http');
const cloneResponse = require('clone-response');
http.get('http://example.com', response => {
const clonedResponse = cloneResponse(response);
response.pipe(process.stdout);
setImmediate(() => {
// The response stream has already been consumed by the time this executes,
// however the cloned response stream is still available.
doSomethingWithResponse(clonedResponse);
});
});
```
Please bear in mind that the process of cloning a stream consumes it. However, you can consume a stream multiple times in the same tick, therefore allowing you to create multiple clones. e.g:
```js
const clone1 = cloneResponse(response);
const clone2 = cloneResponse(response);
// response can still be consumed in this tick but cannot be consumed if passed
// into any async callbacks. clone1 and clone2 can be passed around and be
// consumed in the future.
```
## API
### cloneResponse(response)
Returns a clone of the passed in response.
#### response
Type: `stream`
A [Node.js HTTP response stream](https://nodejs.org/api/http.html#http_class_http_incomingmessage) to clone.
## License
MIT © Luke Childs
{
"name": "clone-response",
"version": "1.0.2",
"description": "Clone a Node.js HTTP response stream",
"main": "src/index.js",
"scripts": {
"test": "xo && nyc ava",
"coverage": "nyc report --reporter=text-lcov | coveralls"
},
"xo": {
"extends": "xo-lukechilds"
},
"repository": {
"type": "git",
"url": "git+https://github.com/lukechilds/clone-response.git"
},
"keywords": [
"clone",
"duplicate",
"copy",
"response",
"HTTP",
"stream"
],
"author": "Luke Childs <lukechilds123@gmail.com> (http://lukechilds.co.uk)",
"license": "MIT",
"bugs": {
"url": "https://github.com/lukechilds/clone-response/issues"
},
"homepage": "https://github.com/lukechilds/clone-response",
"dependencies": {
"mimic-response": "^1.0.0"
},
"devDependencies": {
"ava": "^0.22.0",
"coveralls": "^2.13.1",
"create-test-server": "^2.0.1",
"eslint-config-xo-lukechilds": "^1.0.0",
"get-stream": "^3.0.0",
"nyc": "^11.0.2",
"pify": "^3.0.0",
"xo": "^0.19.0"
}
}
'use strict';
const PassThrough = require('stream').PassThrough;
const mimicResponse = require('mimic-response');
const cloneResponse = response => {
if (!(response && response.pipe)) {
throw new TypeError('Parameter `response` must be a response stream.');
}
const clone = new PassThrough();
mimicResponse(response, clone);
return response.pipe(clone);
};
module.exports = cloneResponse;
'use strict';
const path = require('path');
const os = require('os');
const fs = require('graceful-fs');
const makeDir = require('make-dir');
const xdgBasedir = require('xdg-basedir');
const writeFileAtomic = require('write-file-atomic');
const dotProp = require('dot-prop');
const uniqueString = require('unique-string');
const configDir = xdgBasedir.config || path.join(os.tmpdir(), uniqueString());
const permissionError = 'You don\'t have access to this file.';
const makeDirOptions = {mode: 0o0700};
const writeFileOptions = {mode: 0o0600};
class Configstore {
constructor(id, defaults, options = {}) {
const pathPrefix = options.globalConfigPath ?
path.join(id, 'config.json') :
path.join('configstore', `${id}.json`);
this.path = options.configPath || path.join(configDir, pathPrefix);
if (defaults) {
this.all = Object.assign({}, defaults, this.all);
}
}
get all() {
try {
return JSON.parse(fs.readFileSync(this.path, 'utf8'));
} catch (error) {
// Create directory if it doesn't exist
if (error.code === 'ENOENT') {
return {};
}
// Improve the message of permission errors
if (error.code === 'EACCES') {
error.message = `${error.message}\n${permissionError}\n`;
}
// Empty the file if it encounters invalid JSON
if (error.name === 'SyntaxError') {
writeFileAtomic.sync(this.path, '', writeFileOptions);
return {};
}
throw error;
}
}
set all(value) {
try {
// Make sure the folder exists as it could have been deleted in the meantime
makeDir.sync(path.dirname(this.path), makeDirOptions);
writeFileAtomic.sync(this.path, JSON.stringify(value, null, '\t'), writeFileOptions);
} catch (error) {
// Improve the message of permission errors
if (error.code === 'EACCES') {
error.message = `${error.message}\n${permissionError}\n`;
}
throw error;
}
}
get size() {
return Object.keys(this.all || {}).length;
}
get(key) {
return dotProp.get(this.all, key);
}
set(key, value) {
const config = this.all;
if (arguments.length === 1) {
for (const k of Object.keys(key)) {
dotProp.set(config, k, key[k]);
}
} else {
dotProp.set(config, key, value);
}
this.all = config;
}
has(key) {
return dotProp.has(this.all, key);
}
delete(key) {
const config = this.all;
dotProp.delete(config, key);
this.all = config;
}
clear() {
this.all = {};
}
}
module.exports = Configstore;
BSD 2-Clause License
Copyright (c) Google
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
{
"name": "configstore",
"version": "4.0.0",
"description": "Easily load and save config without having to think about where and how",
"license": "BSD-2-Clause",
"repository": "yeoman/configstore",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"engines": {
"node": ">=6"
},
"scripts": {
"test": "xo && ava"
},
"files": [
"index.js"
],
"keywords": [
"config",
"store",
"storage",
"conf",
"configuration",
"settings",
"preferences",
"json",
"data",
"persist",
"persistent",
"save"
],
"dependencies": {
"dot-prop": "^4.1.0",
"graceful-fs": "^4.1.2",
"make-dir": "^1.0.0",
"unique-string": "^1.0.0",
"write-file-atomic": "^2.0.0",
"xdg-basedir": "^3.0.0"
},
"devDependencies": {
"ava": "*",
"xo": "*"
}
}
# configstore [![Build Status](https://travis-ci.org/yeoman/configstore.svg?branch=master)](https://travis-ci.org/yeoman/configstore)
> Easily load and persist config without having to think about where and how
Config is stored in a JSON file located in `$XDG_CONFIG_HOME` or `~/.config`.<br>
Example: `~/.config/configstore/some-id.json`
*If you need this for Electron, check out [`electron-store`](https://github.com/sindresorhus/electron-store) instead.*
## Install
```
$ npm install configstore
```
## Usage
```js
const Configstore = require('configstore');
const pkg = require('./package.json');
// create a Configstore instance with an unique ID e.g.
// Package name and optionally some default values
const conf = new Configstore(pkg.name, {foo: 'bar'});
console.log(conf.get('foo'));
//=> 'bar'
conf.set('awesome', true);
console.log(conf.get('awesome'));
//=> true
// Use dot-notation to access nested properties
conf.set('bar.baz', true);
console.log(conf.get('bar'));
//=> {baz: true}
conf.delete('awesome');
console.log(conf.get('awesome'));
//=> undefined
```
## API
### Configstore(packageName, [defaults], [options])
Returns a new instance.
#### packageName
Type: `string`
Name of your package.
#### defaults
Type: `Object`
Default config.
#### options
##### globalConfigPath
Type: `boolean`<br>
Default: `false`
Store the config at `$CONFIG/package-name/config.json` instead of the default `$CONFIG/configstore/package-name.json`. This is not recommended as you might end up conflicting with other tools, rendering the "without having to think" idea moot.
##### configPath
Type: `string`<br>
Default: Automatic
**Please don't use this option unless absolutely necessary and you know what you're doing.**
Set the path of the config file. Overrides the `packageName` and `globalConfigPath` options.
### Instance
You can use [dot-notation](https://github.com/sindresorhus/dot-prop) in a `key` to access nested properties.
### .set(key, value)
Set an item.
### .set(object)
Set multiple items at once.
### .get(key)
Get an item.
### .has(key)
Check if an item exists.
### .delete(key)
Delete an item.
### .clear()
Delete all items.
### .size
Get the item count.
### .path
Get the path to the config file. Can be used to show the user where the config file is located or even better open it for them.
### .all
Get all the config as an object or replace the current config with an object:
```js
conf.all = {
hello: 'world'
};
```
## License
[BSD license](http://opensource.org/licenses/bsd-license.php)<br>
Copyright Google
'use strict';
const crypto = require('crypto');
module.exports = len => {
if (!Number.isFinite(len)) {
throw new TypeError('Expected a finite number');
}
return crypto.randomBytes(Math.ceil(len / 2)).toString('hex').slice(0, len);
};
The MIT License (MIT)
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
{
"name": "crypto-random-string",
"version": "1.0.0",
"description": "Generate a cryptographically strong random string",
"license": "MIT",
"repository": "sindresorhus/crypto-random-string",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"engines": {
"node": ">=4"
},
"scripts": {
"test": "xo && ava"
},
"files": [
"index.js"
],
"keywords": [
"random",
"string",
"str",
"rand",
"text",
"id",
"identifier",
"slug",
"salt",
"crypto",
"strong",
"secure",
"hex"
],
"devDependencies": {
"ava": "*",
"xo": "*"
},
"xo": {
"esnext": true
}
}
# crypto-random-string [![Build Status](https://travis-ci.org/sindresorhus/crypto-random-string.svg?branch=master)](https://travis-ci.org/sindresorhus/crypto-random-string)
> Generate a [cryptographically strong](https://en.m.wikipedia.org/wiki/Strong_cryptography) random string
Can be useful for creating an identifier, slug, salt, fixture, etc.
## Install
```
$ npm install --save crypto-random-string
```
## Usage
```js
const cryptoRandomString = require('crypto-random-string');
cryptoRandomString(10);
//=> '2cf05d94db'
```
## API
### cryptoRandomString(length)
#### length
Type: `number`
Length of the returned string.
## Related
- [random-int](https://github.com/sindresorhus/random-int) - Generate a random integer
- [random-float](https://github.com/sindresorhus/random-float) - Generate a random float
- [random-item](https://github.com/sindresorhus/random-item) - Get a random item from an array
- [random-boolean](https://github.com/arthurvr/random-boolean) - Get a random boolean
- [random-obj-key](https://github.com/sindresorhus/random-obj-key) - Get a random key from an object
- [random-obj-prop](https://github.com/sindresorhus/random-obj-prop) - Get a random property from an object
- [unique-random](https://github.com/sindresorhus/unique-random) - Generate random numbers that are consecutively unique
## License
MIT © [Sindre Sorhus](https://sindresorhus.com)
'use strict';
const PassThrough = require('stream').PassThrough;
const zlib = require('zlib');
const mimicResponse = require('mimic-response');
module.exports = response => {
// TODO: Use Array#includes when targeting Node.js 6
if (['gzip', 'deflate'].indexOf(response.headers['content-encoding']) === -1) {
return response;
}
const unzip = zlib.createUnzip();
const stream = new PassThrough();
mimicResponse(response, stream);
unzip.on('error', err => {
if (err.code === 'Z_BUF_ERROR') {
stream.end();
return;
}
stream.emit('error', err);
});
response.pipe(unzip).pipe(stream);
return stream;
};
`The MIT License (MIT)
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
{
"name": "decompress-response",
"version": "3.3.0",
"description": "Decompress a HTTP response if needed",
"license": "MIT",
"repository": "sindresorhus/decompress-response",
"maintainers": [
{
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
{
"name": "Vsevolod Strukchinsky",
"email": "floatdrop@gmail.com",
"url": "github.com/floatdrop"
}
],
"engines": {
"node": ">=4"
},
"scripts": {
"test": "xo && ava"
},
"files": [
"index.js"
],
"keywords": [
"decompress",
"response",
"http",
"https",
"zlib",
"gzip",
"zip",
"deflate",
"unzip",
"ungzip",
"incoming",
"message",
"stream",
"compressed"
],
"dependencies": {
"mimic-response": "^1.0.0"
},
"devDependencies": {
"ava": "*",
"get-stream": "^3.0.0",
"pify": "^3.0.0",
"xo": "*"
}
}
# decompress-response [![Build Status](https://travis-ci.org/sindresorhus/decompress-response.svg?branch=master)](https://travis-ci.org/sindresorhus/decompress-response)
> Decompress a HTTP response if needed
Decompresses the [response](https://nodejs.org/api/http.html#http_class_http_incomingmessage) from [`http.request`](https://nodejs.org/api/http.html#http_http_request_options_callback) if it's gzipped or deflated, otherwise just passes it through.
Used by [`got`](https://github.com/sindresorhus/got).
## Install
```
$ npm install decompress-response
```
## Usage
```js
const http = require('http');
const decompressResponse = require('decompress-response');
http.get('http://sindresorhus.com', response => {
response = decompressResponse(response);
});
```
## License
MIT © [Sindre Sorhus](https://sindresorhus.com)
MIT License
Copyright (c) 2018 Szymon Marczak
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
# defer-to-connect
> The safe way to handle the `connect` socket event
[![Coverage Status](https://coveralls.io/repos/github/szmarczak/defer-to-connect/badge.svg?branch=master)](https://coveralls.io/github/szmarczak/defer-to-connect?branch=master)
Once you receive the socket, it may be already connected (or disconnected).<br>
To avoid checking that, use `defer-to-connect`. It'll do that for you.
## Usage
```js
const deferToConnect = require('defer-to-connect');
deferToConnect(socket, () => {
console.log('Connected!');
});
```
## API
### deferToConnect(socket, connectListener)
Calls `connectListener()` when connected.
### deferToConnect(socket, listeners)
#### listeners
An object representing `connect`, `secureConnect` and `close` properties.
Calls `connect()` when the socket is connected.<br>
Calls `secureConnect()` when the socket is securely connected.<br>
Calls `close()` when the socket is destroyed.
## License
MIT
/// <reference types="node" />
import { Socket } from 'net';
import { TLSSocket } from 'tls';
interface Listeners {
connect?: () => void;
secureConnect?: () => void;
close?: (hadError: boolean) => void;
}
declare const deferToConnect: (socket: Socket | TLSSocket, fn: Listeners | (() => void)) => void;
export default deferToConnect;
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const tls_1 = require("tls");
const deferToConnect = (socket, fn) => {
let listeners;
if (typeof fn === 'function') {
const connect = fn;
listeners = { connect };
}
else {
listeners = fn;
}
const hasConnectListener = typeof listeners.connect === 'function';
const hasSecureConnectListener = typeof listeners.secureConnect === 'function';
const hasCloseListener = typeof listeners.close === 'function';
const onConnect = () => {
if (hasConnectListener) {
listeners.connect();
}
if (socket instanceof tls_1.TLSSocket && hasSecureConnectListener) {
if (socket.authorized) {
listeners.secureConnect();
}
else if (!socket.authorizationError) {
socket.once('secureConnect', listeners.secureConnect);
}
}
if (hasCloseListener) {
socket.once('close', listeners.close);
}
};
if (socket.writable && !socket.connecting) {
onConnect();
}
else if (socket.connecting) {
socket.once('connect', onConnect);
}
else if (socket.destroyed && hasCloseListener) {
listeners.close(socket._hadError);
}
};
exports.default = deferToConnect;
// For CommonJS default export support
module.exports = deferToConnect;
module.exports.default = deferToConnect;
{
"name": "defer-to-connect",
"version": "1.1.3",
"description": "The safe way to handle the `connect` socket event",
"main": "dist",
"files": [
"dist"
],
"scripts": {
"build": "del-cli dist && tsc",
"prepublishOnly": "npm run build",
"test": "xo && nyc ava",
"coveralls": "nyc report --reporter=text-lcov | coveralls"
},
"keywords": [
"socket",
"connect",
"event"
],
"author": "Szymon Marczak",
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/szmarczak/defer-to-connect.git"
},
"bugs": {
"url": "https://github.com/szmarczak/defer-to-connect/issues"
},
"homepage": "https://github.com/szmarczak/defer-to-connect#readme",
"xo": {
"extends": "xo-typescript",
"extensions": [
"ts"
],
"rules": {
"ava/no-ignored-test-files": "off"
}
},
"devDependencies": {
"@sindresorhus/tsconfig": "^0.5.0",
"@types/node": "^12.12.4",
"@typescript-eslint/eslint-plugin": "^1.11.0",
"@typescript-eslint/parser": "^1.11.0",
"ava": "^2.1.0",
"coveralls": "^3.0.7",
"create-cert": "^1.0.6",
"del-cli": "^3.0.0",
"eslint-config-xo-typescript": "^0.15.0",
"nyc": "^14.0.0",
"p-event": "^4.1.0",
"ts-node": "^8.1.0",
"typescript": "^3.6.4",
"xo": "^0.25.3"
},
"nyc": {
"extension": [
".ts"
]
},
"ava": {
"babel": false,
"compileEnhancements": false,
"extensions": [
"ts"
],
"require": [
"ts-node/register"
],
"files": [
"!dist/tests/test.d.ts"
]
},
"types": "dist"
}
'use strict';
const isObj = require('is-obj');
const disallowedKeys = [
'__proto__',
'prototype',
'constructor'
];
const isValidPath = pathSegments => !pathSegments.some(segment => disallowedKeys.includes(segment));
function getPathSegments(path) {
const pathArr = path.split('.');
const parts = [];
for (let i = 0; i < pathArr.length; i++) {
let p = pathArr[i];
while (p[p.length - 1] === '\\' && pathArr[i + 1] !== undefined) {
p = p.slice(0, -1) + '.';
p += pathArr[++i];
}
parts.push(p);
}
if (!isValidPath(parts)) {
return [];
}
return parts;
}
module.exports = {
get(obj, path, value) {
if (!isObj(obj) || typeof path !== 'string') {
return value === undefined ? obj : value;
}
const pathArr = getPathSegments(path);
if (pathArr.length === 0) {
return;
}
for (let i = 0; i < pathArr.length; i++) {
if (!Object.prototype.propertyIsEnumerable.call(obj, pathArr[i])) {
return value;
}
obj = obj[pathArr[i]];
if (obj === undefined || obj === null) {
// `obj` is either `undefined` or `null` so we want to stop the loop, and
// if this is not the last bit of the path, and
// if it did't return `undefined`
// it would return `null` if `obj` is `null`
// but we want `get({foo: null}, 'foo.bar')` to equal `undefined`, or the supplied value, not `null`
if (i !== pathArr.length - 1) {
return value;
}
break;
}
}
return obj;
},
set(obj, path, value) {
if (!isObj(obj) || typeof path !== 'string') {
return obj;
}
const root = obj;
const pathArr = getPathSegments(path);
if (pathArr.length === 0) {
return;
}
for (let i = 0; i < pathArr.length; i++) {
const p = pathArr[i];
if (!isObj(obj[p])) {
obj[p] = {};
}
if (i === pathArr.length - 1) {
obj[p] = value;
}
obj = obj[p];
}
return root;
},
delete(obj, path) {
if (!isObj(obj) || typeof path !== 'string') {
return;
}
const pathArr = getPathSegments(path);
for (let i = 0; i < pathArr.length; i++) {
const p = pathArr[i];
if (i === pathArr.length - 1) {
delete obj[p];
return;
}
obj = obj[p];
if (!isObj(obj)) {
return;
}
}
},
has(obj, path) {
if (!isObj(obj) || typeof path !== 'string') {
return false;
}
const pathArr = getPathSegments(path);
for (let i = 0; i < pathArr.length; i++) {
if (isObj(obj)) {
if (!(pathArr[i] in obj)) {
return false;
}
obj = obj[pathArr[i]];
} else {
return false;
}
}
return true;
}
};
The MIT License (MIT)
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
{
"name": "dot-prop",
"version": "4.2.1",
"description": "Get, set, or delete a property from a nested object using a dot path",
"license": "MIT",
"repository": "sindresorhus/dot-prop",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"engines": {
"node": ">=4"
},
"scripts": {
"test": "xo && ava",
"bench": "matcha bench.js"
},
"files": [
"index.js"
],
"keywords": [
"obj",
"object",
"prop",
"property",
"dot",
"path",
"get",
"set",
"delete",
"del",
"access",
"notation",
"dotty"
],
"dependencies": {
"is-obj": "^1.0.0"
},
"devDependencies": {
"ava": "1.4.1",
"matcha": "^0.7.0",
"xo": "0.24.0"
},
"xo": {
"esnext": true
}
}
# dot-prop [![Build Status](https://travis-ci.org/sindresorhus/dot-prop.svg?branch=master)](https://travis-ci.org/sindresorhus/dot-prop)
> Get, set, or delete a property from a nested object using a dot path
## Install
```
$ npm install --save dot-prop
```
## Usage
```js
const dotProp = require('dot-prop');
// getter
dotProp.get({foo: {bar: 'unicorn'}}, 'foo.bar');
//=> 'unicorn'
dotProp.get({foo: {bar: 'a'}}, 'foo.notDefined.deep');
//=> undefined
dotProp.get({foo: {bar: 'a'}}, 'foo.notDefined.deep', 'default value');
//=> 'default value'
dotProp.get({foo: {'dot.dot': 'unicorn'}}, 'foo.dot\\.dot');
//=> 'unicorn'
// setter
const obj = {foo: {bar: 'a'}};
dotProp.set(obj, 'foo.bar', 'b');
console.log(obj);
//=> {foo: {bar: 'b'}}
const foo = dotProp.set({}, 'foo.bar', 'c');
console.log(foo);
//=> {foo: {bar: 'c'}}
dotProp.set(obj, 'foo.baz', 'x');
console.log(obj);
//=> {foo: {bar: 'b', baz: 'x'}}
// has
dotProp.has({foo: {bar: 'unicorn'}}, 'foo.bar');
//=> true
// deleter
const obj = {foo: {bar: 'a'}};
dotProp.delete(obj, 'foo.bar');
console.log(obj);
//=> {foo: {}}
obj.foo.bar = {x: 'y', y: 'x'};
dotProp.delete(obj, 'foo.bar.x');
console.log(obj);
//=> {foo: {bar: {y: 'x'}}}
```
## API
### get(obj, path, [defaultValue])
### set(obj, path, value)
Returns the object.
### has(obj, path)
### delete(obj, path)
#### obj
Type: `Object`
Object to get, set, or delete the `path` value.
#### path
Type: `string`
Path of the property in the object, using `.` to separate each nested key.
Use `\\.` if you have a `.` in the key.
The following path components are invalid and results in `undefined` being returned: `__proto__`, `prototype`, `constructor`.
#### value
Type: `any`
Value to set at `path`.
#### defaultValue
Type: `any`
Default value.
## License
MIT © [Sindre Sorhus](https://sindresorhus.com)
Copyright (c) 2013, Deoxxa Development
======================================
All rights reserved.
--------------------
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of Deoxxa Development nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY DEOXXA DEVELOPMENT ''AS IS'' AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL DEOXXA DEVELOPMENT BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# duplexer3 [![Build Status](https://travis-ci.org/floatdrop/duplexer3.svg?branch=master)](https://travis-ci.org/floatdrop/duplexer3) [![Coverage Status](https://coveralls.io/repos/floatdrop/duplexer3/badge.svg?branch=master&service=github)](https://coveralls.io/github/floatdrop/duplexer3?branch=master)
Like [duplexer2](https://github.com/deoxxa/duplexer2) but using Streams3 without readable-stream dependency
```javascript
var stream = require("stream");
var duplexer3 = require("duplexer3");
var writable = new stream.Writable({objectMode: true}),
readable = new stream.Readable({objectMode: true});
writable._write = function _write(input, encoding, done) {
if (readable.push(input)) {
return done();
} else {
readable.once("drain", done);
}
};
readable._read = function _read(n) {
// no-op
};
// simulate the readable thing closing after a bit
writable.once("finish", function() {
setTimeout(function() {
readable.push(null);
}, 500);
});
var duplex = duplexer3(writable, readable);
duplex.on("data", function(e) {
console.log("got data", JSON.stringify(e));
});
duplex.on("finish", function() {
console.log("got finish event");
});
duplex.on("end", function() {
console.log("got end event");
});
duplex.write("oh, hi there", function() {
console.log("finished writing");
});
duplex.end(function() {
console.log("finished ending");
});
```
```
got data "oh, hi there"
finished writing
got finish event
finished ending
got end event
```
## Overview
This is a reimplementation of [duplexer](https://www.npmjs.com/package/duplexer) using the
Streams3 API which is standard in Node as of v4. Everything largely
works the same.
## Installation
[Available via `npm`](https://docs.npmjs.com/cli/install):
```
$ npm i duplexer3
```
## API
### duplexer3
Creates a new `DuplexWrapper` object, which is the actual class that implements
most of the fun stuff. All that fun stuff is hidden. DON'T LOOK.
```javascript
duplexer3([options], writable, readable)
```
```javascript
const duplex = duplexer3(new stream.Writable(), new stream.Readable());
```
Arguments
* __options__ - an object specifying the regular `stream.Duplex` options, as
well as the properties described below.
* __writable__ - a writable stream
* __readable__ - a readable stream
Options
* __bubbleErrors__ - a boolean value that specifies whether to bubble errors
from the underlying readable/writable streams. Default is `true`.
## License
3-clause BSD. [A copy](./LICENSE) is included with the source.
## Contact
* GitHub ([deoxxa](http://github.com/deoxxa))
* Twitter ([@deoxxa](http://twitter.com/deoxxa))
* Email ([deoxxa@fknsrs.biz](mailto:deoxxa@fknsrs.biz))
"use strict";
var stream = require("stream");
function DuplexWrapper(options, writable, readable) {
if (typeof readable === "undefined") {
readable = writable;
writable = options;
options = null;
}
stream.Duplex.call(this, options);
if (typeof readable.read !== "function") {
readable = (new stream.Readable(options)).wrap(readable);
}
this._writable = writable;
this._readable = readable;
this._waiting = false;
var self = this;
writable.once("finish", function() {
self.end();
});
this.once("finish", function() {
writable.end();
});
readable.on("readable", function() {
if (self._waiting) {
self._waiting = false;
self._read();
}
});
readable.once("end", function() {
self.push(null);
});
if (!options || typeof options.bubbleErrors === "undefined" || options.bubbleErrors) {
writable.on("error", function(err) {
self.emit("error", err);
});
readable.on("error", function(err) {
self.emit("error", err);
});
}
}
DuplexWrapper.prototype = Object.create(stream.Duplex.prototype, {constructor: {value: DuplexWrapper}});
DuplexWrapper.prototype._write = function _write(input, encoding, done) {
this._writable.write(input, encoding, done);
};
DuplexWrapper.prototype._read = function _read() {
var buf;
var reads = 0;
while ((buf = this._readable.read()) !== null) {
this.push(buf);
reads++;
}
if (reads === 0) {
this._waiting = true;
}
};
module.exports = function duplex2(options, writable, readable) {
return new DuplexWrapper(options, writable, readable);
};
module.exports.DuplexWrapper = DuplexWrapper;
{
"name": "duplexer3",
"version": "0.1.4",
"description": "Like duplexer but using streams3",
"engine": {
"node": ">=4"
},
"files": [
"index.js"
],
"scripts": {
"test": "mocha -R tap"
},
"repository": "floatdrop/duplexer3",
"keywords": [
"duplex",
"duplexer",
"stream",
"stream3",
"join",
"combine"
],
"author": "Conrad Pankoff <deoxxa@fknsrs.biz> (http://www.fknsrs.biz/)",
"license": "BSD-3-Clause",
"devDependencies": {
"mocha": "^2.2.5"
}
}
The MIT License (MIT)
Copyright (c) 2014 Mathias Buus
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
\ No newline at end of file
# end-of-stream
A node module that calls a callback when a readable/writable/duplex stream has completed or failed.
npm install end-of-stream
[![Build status](https://travis-ci.org/mafintosh/end-of-stream.svg?branch=master)](https://travis-ci.org/mafintosh/end-of-stream)
## Usage
Simply pass a stream and a callback to the `eos`.
Both legacy streams, streams2 and stream3 are supported.
``` js
var eos = require('end-of-stream');
eos(readableStream, function(err) {
// this will be set to the stream instance
if (err) return console.log('stream had an error or closed early');
console.log('stream has ended', this === readableStream);
});
eos(writableStream, function(err) {
if (err) return console.log('stream had an error or closed early');
console.log('stream has finished', this === writableStream);
});
eos(duplexStream, function(err) {
if (err) return console.log('stream had an error or closed early');
console.log('stream has ended and finished', this === duplexStream);
});
eos(duplexStream, {readable:false}, function(err) {
if (err) return console.log('stream had an error or closed early');
console.log('stream has finished but might still be readable');
});
eos(duplexStream, {writable:false}, function(err) {
if (err) return console.log('stream had an error or closed early');
console.log('stream has ended but might still be writable');
});
eos(readableStream, {error:false}, function(err) {
// do not treat emit('error', err) as a end-of-stream
});
```
## License
MIT
## Related
`end-of-stream` is part of the [mississippi stream utility collection](https://github.com/maxogden/mississippi) which includes more useful stream modules similar to this one.
var once = require('once');
var noop = function() {};
var isRequest = function(stream) {
return stream.setHeader && typeof stream.abort === 'function';
};
var isChildProcess = function(stream) {
return stream.stdio && Array.isArray(stream.stdio) && stream.stdio.length === 3
};
var eos = function(stream, opts, callback) {
if (typeof opts === 'function') return eos(stream, null, opts);
if (!opts) opts = {};
callback = once(callback || noop);
var ws = stream._writableState;
var rs = stream._readableState;
var readable = opts.readable || (opts.readable !== false && stream.readable);
var writable = opts.writable || (opts.writable !== false && stream.writable);
var cancelled = false;
var onlegacyfinish = function() {
if (!stream.writable) onfinish();
};
var onfinish = function() {
writable = false;
if (!readable) callback.call(stream);
};
var onend = function() {
readable = false;
if (!writable) callback.call(stream);
};
var onexit = function(exitCode) {
callback.call(stream, exitCode ? new Error('exited with error code: ' + exitCode) : null);
};
var onerror = function(err) {
callback.call(stream, err);
};
var onclose = function() {
process.nextTick(onclosenexttick);
};
var onclosenexttick = function() {
if (cancelled) return;
if (readable && !(rs && (rs.ended && !rs.destroyed))) return callback.call(stream, new Error('premature close'));
if (writable && !(ws && (ws.ended && !ws.destroyed))) return callback.call(stream, new Error('premature close'));
};
var onrequest = function() {
stream.req.on('finish', onfinish);
};
if (isRequest(stream)) {
stream.on('complete', onfinish);
stream.on('abort', onclose);
if (stream.req) onrequest();
else stream.on('request', onrequest);
} else if (writable && !ws) { // legacy streams
stream.on('end', onlegacyfinish);
stream.on('close', onlegacyfinish);
}
if (isChildProcess(stream)) stream.on('exit', onexit);
stream.on('end', onend);
stream.on('finish', onfinish);
if (opts.error !== false) stream.on('error', onerror);
stream.on('close', onclose);
return function() {
cancelled = true;
stream.removeListener('complete', onfinish);
stream.removeListener('abort', onclose);
stream.removeListener('request', onrequest);
if (stream.req) stream.req.removeListener('finish', onfinish);
stream.removeListener('end', onlegacyfinish);
stream.removeListener('close', onlegacyfinish);
stream.removeListener('finish', onfinish);
stream.removeListener('exit', onexit);
stream.removeListener('end', onend);
stream.removeListener('error', onerror);
stream.removeListener('close', onclose);
};
};
module.exports = eos;
{
"name": "end-of-stream",
"version": "1.4.4",
"description": "Call a callback when a readable/writable/duplex stream has completed or failed.",
"repository": {
"type": "git",
"url": "git://github.com/mafintosh/end-of-stream.git"
},
"dependencies": {
"once": "^1.4.0"
},
"scripts": {
"test": "node test.js"
},
"files": [
"index.js"
],
"keywords": [
"stream",
"streams",
"callback",
"finish",
"close",
"end",
"wait"
],
"bugs": {
"url": "https://github.com/mafintosh/end-of-stream/issues"
},
"homepage": "https://github.com/mafintosh/end-of-stream",
"main": "index.js",
"author": "Mathias Buus <mathiasbuus@gmail.com>",
"license": "MIT",
"devDependencies": {
"tape": "^4.11.0"
}
}
'use strict';
const {PassThrough} = require('stream');
module.exports = options => {
options = Object.assign({}, options);
const {array} = options;
let {encoding} = options;
const buffer = encoding === 'buffer';
let objectMode = false;
if (array) {
objectMode = !(encoding || buffer);
} else {
encoding = encoding || 'utf8';
}
if (buffer) {
encoding = null;
}
let len = 0;
const ret = [];
const stream = new PassThrough({objectMode});
if (encoding) {
stream.setEncoding(encoding);
}
stream.on('data', chunk => {
ret.push(chunk);
if (objectMode) {
len = ret.length;
} else {
len += chunk.length;
}
});
stream.getBufferedValue = () => {
if (array) {
return ret;
}
return buffer ? Buffer.concat(ret, len) : ret.join('');
};
stream.getBufferedLength = () => len;
return stream;
};
'use strict';
const pump = require('pump');
const bufferStream = require('./buffer-stream');
class MaxBufferError extends Error {
constructor() {
super('maxBuffer exceeded');
this.name = 'MaxBufferError';
}
}
function getStream(inputStream, options) {
if (!inputStream) {
return Promise.reject(new Error('Expected a stream'));
}
options = Object.assign({maxBuffer: Infinity}, options);
const {maxBuffer} = options;
let stream;
return new Promise((resolve, reject) => {
const rejectPromise = error => {
if (error) { // A null check
error.bufferedData = stream.getBufferedValue();
}
reject(error);
};
stream = pump(inputStream, bufferStream(options), error => {
if (error) {
rejectPromise(error);
return;
}
resolve();
});
stream.on('data', () => {
if (stream.getBufferedLength() > maxBuffer) {
rejectPromise(new MaxBufferError());
}
});
}).then(() => stream.getBufferedValue());
}
module.exports = getStream;
module.exports.buffer = (stream, options) => getStream(stream, Object.assign({}, options, {encoding: 'buffer'}));
module.exports.array = (stream, options) => getStream(stream, Object.assign({}, options, {array: true}));
module.exports.MaxBufferError = MaxBufferError;
MIT License
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
{
"name": "get-stream",
"version": "4.1.0",
"description": "Get a stream as a string, buffer, or array",
"license": "MIT",
"repository": "sindresorhus/get-stream",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"engines": {
"node": ">=6"
},
"scripts": {
"test": "xo && ava"
},
"files": [
"index.js",
"buffer-stream.js"
],
"keywords": [
"get",
"stream",
"promise",
"concat",
"string",
"text",
"buffer",
"read",
"data",
"consume",
"readable",
"readablestream",
"array",
"object"
],
"dependencies": {
"pump": "^3.0.0"
},
"devDependencies": {
"ava": "*",
"into-stream": "^3.0.0",
"xo": "*"
}
}
# get-stream [![Build Status](https://travis-ci.org/sindresorhus/get-stream.svg?branch=master)](https://travis-ci.org/sindresorhus/get-stream)
> Get a stream as a string, buffer, or array
## Install
```
$ npm install get-stream
```
## Usage
```js
const fs = require('fs');
const getStream = require('get-stream');
(async () => {
const stream = fs.createReadStream('unicorn.txt');
console.log(await getStream(stream));
/*
,,))))))));,
__)))))))))))))),
\|/ -\(((((''''((((((((.
-*-==//////(('' . `)))))),
/|\ ))| o ;-. '((((( ,(,
( `| / ) ;))))' ,_))^;(~
| | | ,))((((_ _____------~~~-. %,;(;(>';'~
o_); ; )))(((` ~---~ `:: \ %%~~)(v;(`('~
; ''''```` `: `:::|\,__,%% );`'; ~
| _ ) / `:|`----' `-'
______/\/~ | / /
/~;;.____/;;' / ___--,-( `;;;/
/ // _;______;'------~~~~~ /;;/\ /
// | | / ; \;;,\
(<_ | ; /',/-----' _>
\_| ||_ //~;~~~~~~~~~
`\_| (,~~
\~\
~~
*/
})();
```
## API
The methods returns a promise that resolves when the `end` event fires on the stream, indicating that there is no more data to be read. The stream is switched to flowing mode.
### getStream(stream, [options])
Get the `stream` as a string.
#### options
Type: `Object`
##### encoding
Type: `string`<br>
Default: `utf8`
[Encoding](https://nodejs.org/api/buffer.html#buffer_buffer) of the incoming stream.
##### maxBuffer
Type: `number`<br>
Default: `Infinity`
Maximum length of the returned string. If it exceeds this value before the stream ends, the promise will be rejected with a `getStream.MaxBufferError` error.
### getStream.buffer(stream, [options])
Get the `stream` as a buffer.
It honors the `maxBuffer` option as above, but it refers to byte length rather than string length.
### getStream.array(stream, [options])
Get the `stream` as an array of values.
It honors both the `maxBuffer` and `encoding` options. The behavior changes slightly based on the encoding chosen:
- When `encoding` is unset, it assumes an [object mode stream](https://nodesource.com/blog/understanding-object-streams/) and collects values emitted from `stream` unmodified. In this case `maxBuffer` refers to the number of items in the array (not the sum of their sizes).
- When `encoding` is set to `buffer`, it collects an array of buffers. `maxBuffer` refers to the summed byte lengths of every buffer in the array.
- When `encoding` is set to anything else, it collects an array of strings. `maxBuffer` refers to the summed character lengths of every string in the array.
## Errors
If the input stream emits an `error` event, the promise will be rejected with the error. The buffered data will be attached to the `bufferedData` property of the error.
```js
(async () => {
try {
await getStream(streamThatErrorsAtTheEnd('unicorn'));
} catch (error) {
console.log(error.bufferedData);
//=> 'unicorn'
}
})()
```
## FAQ
### How is this different from [`concat-stream`](https://github.com/maxogden/concat-stream)?
This module accepts a stream instead of being one and returns a promise instead of using a callback. The API is simpler and it only supports returning a string, buffer, or array. It doesn't have a fragile type inference. You explicitly choose what you want. And it doesn't depend on the huge `readable-stream` package.
## Related
- [get-stdin](https://github.com/sindresorhus/get-stdin) - Get stdin as a string or buffer
## License
MIT © [Sindre Sorhus](https://sindresorhus.com)
MIT License
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
/// <reference types="node" />
/// <reference lib="es2016" />
/// <reference lib="es2017.sharedmemory" />
/// <reference lib="esnext.asynciterable" />
/// <reference lib="dom" />
declare type TypedArray = Int8Array | Uint8Array | Uint8ClampedArray | Int16Array | Uint16Array | Int32Array | Uint32Array | Float32Array | Float64Array;
declare type Primitive = null | undefined | string | number | boolean | Symbol;
export interface ArrayLike {
length: number;
}
export interface Class<T = unknown> {
new (...args: any[]): T;
}
declare type DomElement = object & {
nodeType: 1;
nodeName: string;
};
declare type NodeStream = object & {
pipe: Function;
};
export declare const enum TypeName {
null = "null",
boolean = "boolean",
undefined = "undefined",
string = "string",
number = "number",
symbol = "symbol",
Function = "Function",
GeneratorFunction = "GeneratorFunction",
AsyncFunction = "AsyncFunction",
Observable = "Observable",
Array = "Array",
Buffer = "Buffer",
Object = "Object",
RegExp = "RegExp",
Date = "Date",
Error = "Error",
Map = "Map",
Set = "Set",
WeakMap = "WeakMap",
WeakSet = "WeakSet",
Int8Array = "Int8Array",
Uint8Array = "Uint8Array",
Uint8ClampedArray = "Uint8ClampedArray",
Int16Array = "Int16Array",
Uint16Array = "Uint16Array",
Int32Array = "Int32Array",
Uint32Array = "Uint32Array",
Float32Array = "Float32Array",
Float64Array = "Float64Array",
ArrayBuffer = "ArrayBuffer",
SharedArrayBuffer = "SharedArrayBuffer",
DataView = "DataView",
Promise = "Promise",
URL = "URL"
}
declare function is(value: unknown): TypeName;
declare namespace is {
const undefined: (value: unknown) => value is undefined;
const string: (value: unknown) => value is string;
const number: (value: unknown) => value is number;
const function_: (value: unknown) => value is Function;
const null_: (value: unknown) => value is null;
const class_: (value: unknown) => value is Class<unknown>;
const boolean: (value: unknown) => value is boolean;
const symbol: (value: unknown) => value is Symbol;
const numericString: (value: unknown) => boolean;
const array: (arg: any) => arg is any[];
const buffer: (input: unknown) => input is Buffer;
const nullOrUndefined: (value: unknown) => value is null | undefined;
const object: (value: unknown) => value is object;
const iterable: (value: unknown) => value is IterableIterator<unknown>;
const asyncIterable: (value: unknown) => value is AsyncIterableIterator<unknown>;
const generator: (value: unknown) => value is Generator;
const nativePromise: (value: unknown) => value is Promise<unknown>;
const promise: (value: unknown) => value is Promise<unknown>;
const generatorFunction: (value: unknown) => value is GeneratorFunction;
const asyncFunction: (value: unknown) => value is Function;
const boundFunction: (value: unknown) => value is Function;
const regExp: (value: unknown) => value is RegExp;
const date: (value: unknown) => value is Date;
const error: (value: unknown) => value is Error;
const map: (value: unknown) => value is Map<unknown, unknown>;
const set: (value: unknown) => value is Set<unknown>;
const weakMap: (value: unknown) => value is WeakMap<object, unknown>;
const weakSet: (value: unknown) => value is WeakSet<object>;
const int8Array: (value: unknown) => value is Int8Array;
const uint8Array: (value: unknown) => value is Uint8Array;
const uint8ClampedArray: (value: unknown) => value is Uint8ClampedArray;
const int16Array: (value: unknown) => value is Int16Array;
const uint16Array: (value: unknown) => value is Uint16Array;
const int32Array: (value: unknown) => value is Int32Array;
const uint32Array: (value: unknown) => value is Uint32Array;
const float32Array: (value: unknown) => value is Float32Array;
const float64Array: (value: unknown) => value is Float64Array;
const arrayBuffer: (value: unknown) => value is ArrayBuffer;
const sharedArrayBuffer: (value: unknown) => value is SharedArrayBuffer;
const dataView: (value: unknown) => value is DataView;
const directInstanceOf: <T>(instance: unknown, klass: Class<T>) => instance is T;
const urlInstance: (value: unknown) => value is URL;
const urlString: (value: unknown) => boolean;
const truthy: (value: unknown) => boolean;
const falsy: (value: unknown) => boolean;
const nan: (value: unknown) => boolean;
const primitive: (value: unknown) => value is Primitive;
const integer: (value: unknown) => value is number;
const safeInteger: (value: unknown) => value is number;
const plainObject: (value: unknown) => boolean;
const typedArray: (value: unknown) => value is TypedArray;
const arrayLike: (value: unknown) => value is ArrayLike;
const inRange: (value: number, range: number | number[]) => boolean;
const domElement: (value: unknown) => value is DomElement;
const observable: (value: unknown) => boolean;
const nodeStream: (value: unknown) => value is NodeStream;
const infinite: (value: unknown) => boolean;
const even: (value: number) => boolean;
const odd: (value: number) => boolean;
const emptyArray: (value: unknown) => boolean;
const nonEmptyArray: (value: unknown) => boolean;
const emptyString: (value: unknown) => boolean;
const nonEmptyString: (value: unknown) => boolean;
const emptyStringOrWhitespace: (value: unknown) => boolean;
const emptyObject: (value: unknown) => boolean;
const nonEmptyObject: (value: unknown) => boolean;
const emptySet: (value: unknown) => boolean;
const nonEmptySet: (value: unknown) => boolean;
const emptyMap: (value: unknown) => boolean;
const nonEmptyMap: (value: unknown) => boolean;
const any: (predicate: unknown, ...values: unknown[]) => boolean;
const all: (predicate: unknown, ...values: unknown[]) => boolean;
}
export default is;
MIT License
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
{
"name": "@sindresorhus/is",
"version": "0.14.0",
"description": "Type check values: `is.string('🦄') //=> true`",
"license": "MIT",
"repository": "sindresorhus/is",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"main": "dist/index.js",
"engines": {
"node": ">=6"
},
"scripts": {
"lint": "tslint --format stylish --project .",
"build": "del dist && tsc",
"test": "npm run lint && npm run build && ava dist/tests",
"prepublish": "npm run build && del dist/tests"
},
"files": [
"dist"
],
"keywords": [
"type",
"types",
"is",
"check",
"checking",
"validate",
"validation",
"utility",
"util",
"typeof",
"instanceof",
"object",
"assert",
"assertion",
"test",
"kind",
"primitive",
"verify",
"compare"
],
"devDependencies": {
"@sindresorhus/tsconfig": "^0.1.0",
"@types/jsdom": "^11.12.0",
"@types/node": "^10.12.10",
"@types/tempy": "^0.2.0",
"@types/zen-observable": "^0.8.0",
"ava": "^0.25.0",
"del-cli": "^1.1.0",
"jsdom": "^11.6.2",
"rxjs": "^6.3.3",
"tempy": "^0.2.1",
"tslint": "^5.9.1",
"tslint-xo": "^0.10.0",
"typescript": "^3.2.1",
"zen-observable": "^0.8.8"
},
"types": "dist/index.d.ts"
}
This diff is collapsed. Click to expand it.
{
"name": "got",
"version": "9.6.0",
"description": "Simplified HTTP requests",
"license": "MIT",
"repository": "sindresorhus/got",
"main": "source",
"engines": {
"node": ">=8.6"
},
"scripts": {
"test": "xo && nyc ava",
"release": "np"
},
"files": [
"source"
],
"keywords": [
"http",
"https",
"get",
"got",
"url",
"uri",
"request",
"util",
"utility",
"simple",
"curl",
"wget",
"fetch",
"net",
"network",
"electron"
],
"dependencies": {
"@sindresorhus/is": "^0.14.0",
"@szmarczak/http-timer": "^1.1.2",
"cacheable-request": "^6.0.0",
"decompress-response": "^3.3.0",
"duplexer3": "^0.1.4",
"get-stream": "^4.1.0",
"lowercase-keys": "^1.0.1",
"mimic-response": "^1.0.1",
"p-cancelable": "^1.0.0",
"to-readable-stream": "^1.0.0",
"url-parse-lax": "^3.0.0"
},
"devDependencies": {
"ava": "^1.1.0",
"coveralls": "^3.0.0",
"delay": "^4.1.0",
"form-data": "^2.3.3",
"get-port": "^4.0.0",
"np": "^3.1.0",
"nyc": "^13.1.0",
"p-event": "^2.1.0",
"pem": "^1.13.2",
"proxyquire": "^2.0.1",
"sinon": "^7.2.2",
"slow-stream": "0.0.4",
"tempfile": "^2.0.0",
"tempy": "^0.2.1",
"tough-cookie": "^3.0.0",
"xo": "^0.24.0"
},
"ava": {
"concurrency": 4
},
"browser": {
"decompress-response": false,
"electron": false
}
}
This diff is collapsed. Click to expand it.
'use strict';
const EventEmitter = require('events');
const getStream = require('get-stream');
const is = require('@sindresorhus/is');
const PCancelable = require('p-cancelable');
const requestAsEventEmitter = require('./request-as-event-emitter');
const {HTTPError, ParseError, ReadError} = require('./errors');
const {options: mergeOptions} = require('./merge');
const {reNormalize} = require('./normalize-arguments');
const asPromise = options => {
const proxy = new EventEmitter();
const promise = new PCancelable((resolve, reject, onCancel) => {
const emitter = requestAsEventEmitter(options);
onCancel(emitter.abort);
emitter.on('response', async response => {
proxy.emit('response', response);
const stream = is.null(options.encoding) ? getStream.buffer(response) : getStream(response, options);
let data;
try {
data = await stream;
} catch (error) {
reject(new ReadError(error, options));
return;
}
const limitStatusCode = options.followRedirect ? 299 : 399;
response.body = data;
try {
for (const [index, hook] of Object.entries(options.hooks.afterResponse)) {
// eslint-disable-next-line no-await-in-loop
response = await hook(response, updatedOptions => {
updatedOptions = reNormalize(mergeOptions(options, {
...updatedOptions,
retry: 0,
throwHttpErrors: false
}));
// Remove any further hooks for that request, because we we'll call them anyway.
// The loop continues. We don't want duplicates (asPromise recursion).
updatedOptions.hooks.afterResponse = options.hooks.afterResponse.slice(0, index);
return asPromise(updatedOptions);
});
}
} catch (error) {
reject(error);
return;
}
const {statusCode} = response;
if (options.json && response.body) {
try {
response.body = JSON.parse(response.body);
} catch (error) {
if (statusCode >= 200 && statusCode < 300) {
const parseError = new ParseError(error, statusCode, options, data);
Object.defineProperty(parseError, 'response', {value: response});
reject(parseError);
return;
}
}
}
if (statusCode !== 304 && (statusCode < 200 || statusCode > limitStatusCode)) {
const error = new HTTPError(response, options);
Object.defineProperty(error, 'response', {value: response});
if (emitter.retry(error) === false) {
if (options.throwHttpErrors) {
reject(error);
return;
}
resolve(response);
}
return;
}
resolve(response);
});
emitter.once('error', reject);
[
'request',
'redirect',
'uploadProgress',
'downloadProgress'
].forEach(event => emitter.on(event, (...args) => proxy.emit(event, ...args)));
});
promise.on = (name, fn) => {
proxy.on(name, fn);
return promise;
};
return promise;
};
module.exports = asPromise;
'use strict';
const {PassThrough} = require('stream');
const duplexer3 = require('duplexer3');
const requestAsEventEmitter = require('./request-as-event-emitter');
const {HTTPError, ReadError} = require('./errors');
module.exports = options => {
const input = new PassThrough();
const output = new PassThrough();
const proxy = duplexer3(input, output);
const piped = new Set();
let isFinished = false;
options.retry.retries = () => 0;
if (options.body) {
proxy.write = () => {
throw new Error('Got\'s stream is not writable when the `body` option is used');
};
}
const emitter = requestAsEventEmitter(options, input);
// Cancels the request
proxy._destroy = emitter.abort;
emitter.on('response', response => {
const {statusCode} = response;
response.on('error', error => {
proxy.emit('error', new ReadError(error, options));
});
if (options.throwHttpErrors && statusCode !== 304 && (statusCode < 200 || statusCode > 299)) {
proxy.emit('error', new HTTPError(response, options), null, response);
return;
}
isFinished = true;
response.pipe(output);
for (const destination of piped) {
if (destination.headersSent) {
continue;
}
for (const [key, value] of Object.entries(response.headers)) {
// Got gives *decompressed* data. Overriding `content-encoding` header would result in an error.
// It's not possible to decompress already decompressed data, is it?
const allowed = options.decompress ? key !== 'content-encoding' : true;
if (allowed) {
destination.setHeader(key, value);
}
}
destination.statusCode = response.statusCode;
}
proxy.emit('response', response);
});
[
'error',
'request',
'redirect',
'uploadProgress',
'downloadProgress'
].forEach(event => emitter.on(event, (...args) => proxy.emit(event, ...args)));
const pipe = proxy.pipe.bind(proxy);
const unpipe = proxy.unpipe.bind(proxy);
proxy.pipe = (destination, options) => {
if (isFinished) {
throw new Error('Failed to pipe. The response has been emitted already.');
}
const result = pipe(destination, options);
if (Reflect.has(destination, 'setHeader')) {
piped.add(destination);
}
return result;
};
proxy.unpipe = stream => {
piped.delete(stream);
return unpipe(stream);
};
return proxy;
};
'use strict';
const errors = require('./errors');
const asStream = require('./as-stream');
const asPromise = require('./as-promise');
const normalizeArguments = require('./normalize-arguments');
const merge = require('./merge');
const deepFreeze = require('./utils/deep-freeze');
const getPromiseOrStream = options => options.stream ? asStream(options) : asPromise(options);
const aliases = [
'get',
'post',
'put',
'patch',
'head',
'delete'
];
const create = defaults => {
defaults = merge({}, defaults);
normalizeArguments.preNormalize(defaults.options);
if (!defaults.handler) {
// This can't be getPromiseOrStream, because when merging
// the chain would stop at this point and no further handlers would be called.
defaults.handler = (options, next) => next(options);
}
function got(url, options) {
try {
return defaults.handler(normalizeArguments(url, options, defaults), getPromiseOrStream);
} catch (error) {
if (options && options.stream) {
throw error;
} else {
return Promise.reject(error);
}
}
}
got.create = create;
got.extend = options => {
let mutableDefaults;
if (options && Reflect.has(options, 'mutableDefaults')) {
mutableDefaults = options.mutableDefaults;
delete options.mutableDefaults;
} else {
mutableDefaults = defaults.mutableDefaults;
}
return create({
options: merge.options(defaults.options, options),
handler: defaults.handler,
mutableDefaults
});
};
got.mergeInstances = (...args) => create(merge.instances(args));
got.stream = (url, options) => got(url, {...options, stream: true});
for (const method of aliases) {
got[method] = (url, options) => got(url, {...options, method});
got.stream[method] = (url, options) => got.stream(url, {...options, method});
}
Object.assign(got, {...errors, mergeOptions: merge.options});
Object.defineProperty(got, 'defaults', {
value: defaults.mutableDefaults ? defaults : deepFreeze(defaults),
writable: defaults.mutableDefaults,
configurable: defaults.mutableDefaults,
enumerable: true
});
return got;
};
module.exports = create;
'use strict';
const urlLib = require('url');
const http = require('http');
const PCancelable = require('p-cancelable');
const is = require('@sindresorhus/is');
class GotError extends Error {
constructor(message, error, options) {
super(message);
Error.captureStackTrace(this, this.constructor);
this.name = 'GotError';
if (!is.undefined(error.code)) {
this.code = error.code;
}
Object.assign(this, {
host: options.host,
hostname: options.hostname,
method: options.method,
path: options.path,
socketPath: options.socketPath,
protocol: options.protocol,
url: options.href,
gotOptions: options
});
}
}
module.exports.GotError = GotError;
module.exports.CacheError = class extends GotError {
constructor(error, options) {
super(error.message, error, options);
this.name = 'CacheError';
}
};
module.exports.RequestError = class extends GotError {
constructor(error, options) {
super(error.message, error, options);
this.name = 'RequestError';
}
};
module.exports.ReadError = class extends GotError {
constructor(error, options) {
super(error.message, error, options);
this.name = 'ReadError';
}
};
module.exports.ParseError = class extends GotError {
constructor(error, statusCode, options, data) {
super(`${error.message} in "${urlLib.format(options)}": \n${data.slice(0, 77)}...`, error, options);
this.name = 'ParseError';
this.statusCode = statusCode;
this.statusMessage = http.STATUS_CODES[this.statusCode];
}
};
module.exports.HTTPError = class extends GotError {
constructor(response, options) {
const {statusCode} = response;
let {statusMessage} = response;
if (statusMessage) {
statusMessage = statusMessage.replace(/\r?\n/g, ' ').trim();
} else {
statusMessage = http.STATUS_CODES[statusCode];
}
super(`Response code ${statusCode} (${statusMessage})`, {}, options);
this.name = 'HTTPError';
this.statusCode = statusCode;
this.statusMessage = statusMessage;
this.headers = response.headers;
this.body = response.body;
}
};
module.exports.MaxRedirectsError = class extends GotError {
constructor(statusCode, redirectUrls, options) {
super('Redirected 10 times. Aborting.', {}, options);
this.name = 'MaxRedirectsError';
this.statusCode = statusCode;
this.statusMessage = http.STATUS_CODES[this.statusCode];
this.redirectUrls = redirectUrls;
}
};
module.exports.UnsupportedProtocolError = class extends GotError {
constructor(options) {
super(`Unsupported protocol "${options.protocol}"`, {}, options);
this.name = 'UnsupportedProtocolError';
}
};
module.exports.TimeoutError = class extends GotError {
constructor(error, options) {
super(error.message, {code: 'ETIMEDOUT'}, options);
this.name = 'TimeoutError';
this.event = error.event;
}
};
module.exports.CancelError = PCancelable.CancelError;
'use strict';
const decompressResponse = require('decompress-response');
const is = require('@sindresorhus/is');
const mimicResponse = require('mimic-response');
const progress = require('./progress');
module.exports = (response, options, emitter) => {
const downloadBodySize = Number(response.headers['content-length']) || null;
const progressStream = progress.download(response, emitter, downloadBodySize);
mimicResponse(response, progressStream);
const newResponse = options.decompress === true &&
is.function(decompressResponse) &&
options.method !== 'HEAD' ? decompressResponse(progressStream) : progressStream;
if (!options.decompress && ['gzip', 'deflate'].includes(response.headers['content-encoding'])) {
options.encoding = null;
}
emitter.emit('response', newResponse);
emitter.emit('downloadProgress', {
percent: 0,
transferred: 0,
total: downloadBodySize
});
response.pipe(progressStream);
};
'use strict';
const pkg = require('../package.json');
const create = require('./create');
const defaults = {
options: {
retry: {
retries: 2,
methods: [
'GET',
'PUT',
'HEAD',
'DELETE',
'OPTIONS',
'TRACE'
],
statusCodes: [
408,
413,
429,
500,
502,
503,
504
],
errorCodes: [
'ETIMEDOUT',
'ECONNRESET',
'EADDRINUSE',
'ECONNREFUSED',
'EPIPE',
'ENOTFOUND',
'ENETUNREACH',
'EAI_AGAIN'
]
},
headers: {
'user-agent': `${pkg.name}/${pkg.version} (https://github.com/sindresorhus/got)`
},
hooks: {
beforeRequest: [],
beforeRedirect: [],
beforeRetry: [],
afterResponse: []
},
decompress: true,
throwHttpErrors: true,
followRedirect: true,
stream: false,
form: false,
json: false,
cache: false,
useElectronNet: false
},
mutableDefaults: false
};
const got = create(defaults);
module.exports = got;
'use strict';
module.exports = [
'beforeError',
'init',
'beforeRequest',
'beforeRedirect',
'beforeRetry',
'afterResponse'
];
'use strict';
const {URL} = require('url');
const is = require('@sindresorhus/is');
const knownHookEvents = require('./known-hook-events');
const merge = (target, ...sources) => {
for (const source of sources) {
for (const [key, sourceValue] of Object.entries(source)) {
if (is.undefined(sourceValue)) {
continue;
}
const targetValue = target[key];
if (is.urlInstance(targetValue) && (is.urlInstance(sourceValue) || is.string(sourceValue))) {
target[key] = new URL(sourceValue, targetValue);
} else if (is.plainObject(sourceValue)) {
if (is.plainObject(targetValue)) {
target[key] = merge({}, targetValue, sourceValue);
} else {
target[key] = merge({}, sourceValue);
}
} else if (is.array(sourceValue)) {
target[key] = merge([], sourceValue);
} else {
target[key] = sourceValue;
}
}
}
return target;
};
const mergeOptions = (...sources) => {
sources = sources.map(source => source || {});
const merged = merge({}, ...sources);
const hooks = {};
for (const hook of knownHookEvents) {
hooks[hook] = [];
}
for (const source of sources) {
if (source.hooks) {
for (const hook of knownHookEvents) {
hooks[hook] = hooks[hook].concat(source.hooks[hook]);
}
}
}
merged.hooks = hooks;
return merged;
};
const mergeInstances = (instances, methods) => {
const handlers = instances.map(instance => instance.defaults.handler);
const size = instances.length - 1;
return {
methods,
options: mergeOptions(...instances.map(instance => instance.defaults.options)),
handler: (options, next) => {
let iteration = -1;
const iterate = options => handlers[++iteration](options, iteration === size ? next : iterate);
return iterate(options);
}
};
};
module.exports = merge;
module.exports.options = mergeOptions;
module.exports.instances = mergeInstances;
'use strict';
const {URL, URLSearchParams} = require('url'); // TODO: Use the `URL` global when targeting Node.js 10
const urlLib = require('url');
const is = require('@sindresorhus/is');
const urlParseLax = require('url-parse-lax');
const lowercaseKeys = require('lowercase-keys');
const urlToOptions = require('./utils/url-to-options');
const isFormData = require('./utils/is-form-data');
const merge = require('./merge');
const knownHookEvents = require('./known-hook-events');
const retryAfterStatusCodes = new Set([413, 429, 503]);
// `preNormalize` handles static options (e.g. headers).
// For example, when you create a custom instance and make a request
// with no static changes, they won't be normalized again.
//
// `normalize` operates on dynamic options - they cannot be saved.
// For example, `body` is everytime different per request.
// When it's done normalizing the new options, it performs merge()
// on the prenormalized options and the normalized ones.
const preNormalize = (options, defaults) => {
if (is.nullOrUndefined(options.headers)) {
options.headers = {};
} else {
options.headers = lowercaseKeys(options.headers);
}
if (options.baseUrl && !options.baseUrl.toString().endsWith('/')) {
options.baseUrl += '/';
}
if (options.stream) {
options.json = false;
}
if (is.nullOrUndefined(options.hooks)) {
options.hooks = {};
} else if (!is.object(options.hooks)) {
throw new TypeError(`Parameter \`hooks\` must be an object, not ${is(options.hooks)}`);
}
for (const event of knownHookEvents) {
if (is.nullOrUndefined(options.hooks[event])) {
if (defaults) {
options.hooks[event] = [...defaults.hooks[event]];
} else {
options.hooks[event] = [];
}
}
}
if (is.number(options.timeout)) {
options.gotTimeout = {request: options.timeout};
} else if (is.object(options.timeout)) {
options.gotTimeout = options.timeout;
}
delete options.timeout;
const {retry} = options;
options.retry = {
retries: 0,
methods: [],
statusCodes: [],
errorCodes: []
};
if (is.nonEmptyObject(defaults) && retry !== false) {
options.retry = {...defaults.retry};
}
if (retry !== false) {
if (is.number(retry)) {
options.retry.retries = retry;
} else {
options.retry = {...options.retry, ...retry};
}
}
if (options.gotTimeout) {
options.retry.maxRetryAfter = Math.min(...[options.gotTimeout.request, options.gotTimeout.connection].filter(n => !is.nullOrUndefined(n)));
}
if (is.array(options.retry.methods)) {
options.retry.methods = new Set(options.retry.methods.map(method => method.toUpperCase()));
}
if (is.array(options.retry.statusCodes)) {
options.retry.statusCodes = new Set(options.retry.statusCodes);
}
if (is.array(options.retry.errorCodes)) {
options.retry.errorCodes = new Set(options.retry.errorCodes);
}
return options;
};
const normalize = (url, options, defaults) => {
if (is.plainObject(url)) {
options = {...url, ...options};
url = options.url || {};
delete options.url;
}
if (defaults) {
options = merge({}, defaults.options, options ? preNormalize(options, defaults.options) : {});
} else {
options = merge({}, preNormalize(options));
}
if (!is.string(url) && !is.object(url)) {
throw new TypeError(`Parameter \`url\` must be a string or object, not ${is(url)}`);
}
if (is.string(url)) {
if (options.baseUrl) {
if (url.toString().startsWith('/')) {
url = url.toString().slice(1);
}
url = urlToOptions(new URL(url, options.baseUrl));
} else {
url = url.replace(/^unix:/, 'http://$&');
url = urlParseLax(url);
}
} else if (is(url) === 'URL') {
url = urlToOptions(url);
}
// Override both null/undefined with default protocol
options = merge({path: ''}, url, {protocol: url.protocol || 'https:'}, options);
for (const hook of options.hooks.init) {
const called = hook(options);
if (is.promise(called)) {
throw new TypeError('The `init` hook must be a synchronous function');
}
}
const {baseUrl} = options;
Object.defineProperty(options, 'baseUrl', {
set: () => {
throw new Error('Failed to set baseUrl. Options are normalized already.');
},
get: () => baseUrl
});
const {query} = options;
if (is.nonEmptyString(query) || is.nonEmptyObject(query) || query instanceof URLSearchParams) {
if (!is.string(query)) {
options.query = (new URLSearchParams(query)).toString();
}
options.path = `${options.path.split('?')[0]}?${options.query}`;
delete options.query;
}
if (options.hostname === 'unix') {
const matches = /(.+?):(.+)/.exec(options.path);
if (matches) {
const [, socketPath, path] = matches;
options = {
...options,
socketPath,
path,
host: null
};
}
}
const {headers} = options;
for (const [key, value] of Object.entries(headers)) {
if (is.nullOrUndefined(value)) {
delete headers[key];
}
}
if (options.json && is.undefined(headers.accept)) {
headers.accept = 'application/json';
}
if (options.decompress && is.undefined(headers['accept-encoding'])) {
headers['accept-encoding'] = 'gzip, deflate';
}
const {body} = options;
if (is.nullOrUndefined(body)) {
options.method = options.method ? options.method.toUpperCase() : 'GET';
} else {
const isObject = is.object(body) && !is.buffer(body) && !is.nodeStream(body);
if (!is.nodeStream(body) && !is.string(body) && !is.buffer(body) && !(options.form || options.json)) {
throw new TypeError('The `body` option must be a stream.Readable, string or Buffer');
}
if (options.json && !(isObject || is.array(body))) {
throw new TypeError('The `body` option must be an Object or Array when the `json` option is used');
}
if (options.form && !isObject) {
throw new TypeError('The `body` option must be an Object when the `form` option is used');
}
if (isFormData(body)) {
// Special case for https://github.com/form-data/form-data
headers['content-type'] = headers['content-type'] || `multipart/form-data; boundary=${body.getBoundary()}`;
} else if (options.form) {
headers['content-type'] = headers['content-type'] || 'application/x-www-form-urlencoded';
options.body = (new URLSearchParams(body)).toString();
} else if (options.json) {
headers['content-type'] = headers['content-type'] || 'application/json';
options.body = JSON.stringify(body);
}
options.method = options.method ? options.method.toUpperCase() : 'POST';
}
if (!is.function(options.retry.retries)) {
const {retries} = options.retry;
options.retry.retries = (iteration, error) => {
if (iteration > retries) {
return 0;
}
if ((!error || !options.retry.errorCodes.has(error.code)) && (!options.retry.methods.has(error.method) || !options.retry.statusCodes.has(error.statusCode))) {
return 0;
}
if (Reflect.has(error, 'headers') && Reflect.has(error.headers, 'retry-after') && retryAfterStatusCodes.has(error.statusCode)) {
let after = Number(error.headers['retry-after']);
if (is.nan(after)) {
after = Date.parse(error.headers['retry-after']) - Date.now();
} else {
after *= 1000;
}
if (after > options.retry.maxRetryAfter) {
return 0;
}
return after;
}
if (error.statusCode === 413) {
return 0;
}
const noise = Math.random() * 100;
return ((2 ** (iteration - 1)) * 1000) + noise;
};
}
return options;
};
const reNormalize = options => normalize(urlLib.format(options), options);
module.exports = normalize;
module.exports.preNormalize = preNormalize;
module.exports.reNormalize = reNormalize;
'use strict';
const {Transform} = require('stream');
module.exports = {
download(response, emitter, downloadBodySize) {
let downloaded = 0;
return new Transform({
transform(chunk, encoding, callback) {
downloaded += chunk.length;
const percent = downloadBodySize ? downloaded / downloadBodySize : 0;
// Let `flush()` be responsible for emitting the last event
if (percent < 1) {
emitter.emit('downloadProgress', {
percent,
transferred: downloaded,
total: downloadBodySize
});
}
callback(null, chunk);
},
flush(callback) {
emitter.emit('downloadProgress', {
percent: 1,
transferred: downloaded,
total: downloadBodySize
});
callback();
}
});
},
upload(request, emitter, uploadBodySize) {
const uploadEventFrequency = 150;
let uploaded = 0;
let progressInterval;
emitter.emit('uploadProgress', {
percent: 0,
transferred: 0,
total: uploadBodySize
});
request.once('error', () => {
clearInterval(progressInterval);
});
request.once('response', () => {
clearInterval(progressInterval);
emitter.emit('uploadProgress', {
percent: 1,
transferred: uploaded,
total: uploadBodySize
});
});
request.once('socket', socket => {
const onSocketConnect = () => {
progressInterval = setInterval(() => {
const lastUploaded = uploaded;
/* istanbul ignore next: see #490 (occurs randomly!) */
const headersSize = request._header ? Buffer.byteLength(request._header) : 0;
uploaded = socket.bytesWritten - headersSize;
// Don't emit events with unchanged progress and
// prevent last event from being emitted, because
// it's emitted when `response` is emitted
if (uploaded === lastUploaded || uploaded === uploadBodySize) {
return;
}
emitter.emit('uploadProgress', {
percent: uploadBodySize ? uploaded / uploadBodySize : 0,
transferred: uploaded,
total: uploadBodySize
});
}, uploadEventFrequency);
};
/* istanbul ignore next: hard to test */
if (socket.connecting) {
socket.once('connect', onSocketConnect);
} else if (socket.writable) {
// The socket is being reused from pool,
// so the connect event will not be emitted
onSocketConnect();
}
});
}
};
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.
This diff is collapsed. Click to expand it.