Mirrors/CyberChef
2
0
Fork 0
mirror of https://github.com/gchq/CyberChef synced 2024-11-14 08:37:07 +00:00
Code Issues Projects Releases Packages Wiki Activity

Merge remote-tracking branch 'upstream/master' into features/rsa

Browse source
This commit is contained in:
Matt 2019-09-30 11:03:41 +01:00
commit 841e760b04
No known key found for this signature in database
GPG key ID: 2DD462FE98BF38C2
593 changed files with 53203 additions and 8470 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
.babelrc
View file

@ -1,19 +0,0 @@
{
"presets": [
["env", {
"targets": {
"chrome": 40,
"firefox": 35,
"edge": 14,
"node": "6.5"
},
"modules": false,
"useBuiltIns": true
}]
],
"plugins": [
["babel-plugin-transform-builtin-extend", {
"globals": ["Error"]
}]
]
}

2
.editorconfig
View file

@ -9,6 +9,6 @@ trim_trailing_whitespace = true
indent_style = space
indent_size = 4
[{package.json,.travis.yml}]
[{package.json,.travis.yml,nightwatch.json}]
indent_style = space
indent_size = 2

1
.eslintignore
View file

@ -1 +1,2 @@
src/core/vendor/**
src/web/static/clippy_assets/**

22
.eslintrc.json
View file

@ -1,10 +1,12 @@
{
"parser": "babel-eslint",
"parserOptions": {
"ecmaVersion": 9,
"ecmaFeatures": {
"impliedStrict": true
},
"sourceType": "module"
"sourceType": "module",
"allowImportExportEverywhere": true
},
"env": {
"browser": true,
@ -36,9 +38,12 @@
// disable rules from base configurations
"no-control-regex": "off",
"require-atomic-updates": "off",
"no-async-promise-executor": "off",
// stylistic conventions
"brace-style": ["error", "1tbs"],
"space-before-blocks": ["error", "always"],
"block-spacing": "error",
"array-bracket-spacing": "error",
"comma-spacing": "error",
@ -87,16 +92,23 @@
"no-var": "error",
"prefer-const": "error"
},
"overrides": [
{
"files": "tests/**/*",
"rules": {
"no-unused-expressions": "off",
"no-console": "off"
}
}
],
"globals": {
"$": false,
"jQuery": false,
"log": false,
"app": false,
"COMPILE_TIME": false,
"COMPILE_MSG": false,
"PKG_VERSION": false,
"ENVIRONMENT_IS_WORKER": false,
"ENVIRONMENT_IS_NODE": false,
"ENVIRONMENT_IS_WEB": false
"PKG_VERSION": false
}
}

12
.github/CONTRIBUTING.md vendored
View file

@ -6,7 +6,7 @@ There are lots of opportunities to contribute to CyberChef. If you want ideas, t
Before your contributions can be accepted, you must:
- Sign the [GCHQ Contributor Licence Agreement](https://github.com/gchq/Gaffer/wiki/GCHQ-OSS-Contributor-License-Agreement-V1.0)
- Sign the [GCHQ Contributor Licence Agreement](https://cla-assistant.io/gchq/CyberChef)
- Push your changes to your fork.
- Submit a pull request.
@ -22,15 +22,15 @@ Before your contributions can be accepted, you must:
* Line endings: UNIX style (\n)
## Design Principals
## Design Principles
1. If at all possible, all operations and features should be client-side and not rely on connections to an external server. This increases the utility of CyberChef on closed networks and in virtual machines that are not connected to the Internet. Calls to external APIs may be accepted if there is no other option, but not for critical components.
2. Latency should be kept to a minimum to enhance the user experience. This means that all operation code should sit on the client, rather than being loaded dynamically from a server.
3. Use Vanilla JS if at all possible to reduce the number of libraries required and relied upon. Frameworks like jQuery, although included, should not be used unless absolutely necessary.
4. Minimise the use of large libraries, especially for niche operations that won't be used very often - these will be downloaded by everyone using the app, whether they use that operation or not (due to principal 2).
2. Latency should be kept to a minimum to enhance the user experience. This means that operation code should sit on the client and be executed there. However, as a trade-off between latency and bandwidth, operation code with large dependencies can be loaded in discrete modules in order to reduce the size of the initial download. The downloading of additional modules must remain entirely transparent so that the user is not inconvenienced.
3. Large libraries should be kept in separate modules so that they are not downloaded by everyone who uses the app, just those who specifically require the relevant operations.
4. Use Vanilla JS if at all possible to reduce the number of libraries required and relied upon. Frameworks like jQuery, although included, should not be used unless absolutely necessary.
With these principals in mind, any changes or additions to CyberChef should keep it:
With these principles in mind, any changes or additions to CyberChef should keep it:
- Standalone
- Efficient

15
.github/ISSUE_TEMPLATE.md vendored
View file

@ -1,14 +1 @@
<!-- Prefix the title above with one of the following: -->
<!-- Bug report: -->
<!-- Operation request: -->
<!-- Feature request: -->
<!-- Misc: -->
### Summary
### Example
<!-- If describing a bug, tell us what happens instead of the expected behavior -->
<!-- Include a link that triggers the bug if possible -->
<!-- If you are requesting a new operation, include example input and output -->
<!-- Prefix the title above with 'Misc:' -->

33
.github/ISSUE_TEMPLATE/bug-report.md vendored Normal file
View file

@ -0,0 +1,33 @@
---
name: Bug report
about: Create a report to help us improve
title: 'Bug report: <Insert title here>'
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behaviour or a link to the recipe / input used to cause the bug:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behaviour**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (if relevant, please complete the following information):**
- OS: [e.g. Windows]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

34
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,34 @@
---
name: Bug report
about: Create a report to help us improve
title: 'Bug report: <Insert title here>'
labels: bug
assignees: ''
---
<!-- Prefix the title above with 'Bug report:' -->
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior or a link to the recipe / input used to cause the bug:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (if relevant, please complete the following information):**
- OS: [e.g. Windows]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

20
.github/ISSUE_TEMPLATE/feature_request.md vendored Normal file
View file

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for the project
title: 'Feature request: <Insert title here>'
labels: feature
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. E.g. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

14
.github/ISSUE_TEMPLATE/operation-request.md vendored Normal file
View file

@ -0,0 +1,14 @@
---
name: Operation request
about: Suggest a new operation
title: 'Operation request: <Insert title here>'
labels: operation
assignees: ''
---
## Summary
### Example Input
### Example Output

9
.gitignore vendored
View file

@ -2,11 +2,14 @@ node_modules
npm-debug.log
travis.log
build
docs/*
!docs/*.conf.json
!docs/*.ico
.vscode
.*.swp
.DS_Store
src/core/config/modules/*
src/core/config/OperationConfig.json
src/core/operations/index.mjs
src/node/config/OperationConfig.json
src/node/index.mjs
**/*.DS_Store
tests/browser/output/*

1
.npmignore
View file

@ -3,6 +3,5 @@ npm-debug.log
travis.log
build/*
!build/node
docs
.vscode
.github

17
.travis.yml
View file

@ -1,15 +1,19 @@
language: node_js
node_js:
- node
- lts/*
cache: npm
addons:
chrome: stable
install: npm install
before_script:
- npm install -g grunt
- export NODE_OPTIONS=--max_old_space_size=2048
script:
- grunt lint
- grunt test
- grunt docs
- grunt node
- grunt testnodeconsumer
- grunt prod --msg="$COMPILE_MSG"
- xvfb-run --server-args="-screen 0 1200x800x24" grunt testui
before_deploy:
- grunt exec:sitemap
- grunt copy:ghPages
@ -26,9 +30,10 @@ deploy:
skip_cleanup: true
api_key:
secure: "HV1WSKv4l/0Y2bKKs1iBJocBcmLj08PCRUeEM/jTwA4jqJ8EiLHWiXtER/D5sEg2iibRVKd2OQjfrmS6bo4AiwdeVgAKmv0FtS2Jw+391N8Nd5AkEANHa5Om/IpHLTL2YRAjpJTsDpY72bMUTJIwjQA3TFJkgrpOw6KYfohOcgbxLpZ4XuNJRU3VL4Hsxdv5V9aOVmfFOmMOVPQlakXy7NgtW5POp1f2WJwgcZxylkR1CjwaqMyXmSoVl46pyH3tr5+dptsQoKSGdi6sIHGA60oDotFPcm+0ifa47wZw+vapuuDi4tdNxhrHGaDMG8xiE0WFDHwQUDlk2/+W7j9SEX0H3Em7us371JXRp56EDwEcDa34VpVkC6i8HGcHK55hnxVbMZXGf3qhOFD8wY7qMbjMRvIpucrMHBi86OfkDfv0vDj2LyvIl5APj/AX50BrE0tfH1MZbH26Jkx4NdlkcxQ14GumarmUqfmVvbX/fsoA6oUuAAE9ZgRRi3KHO4wci6KUcRfdm+XOeUkaBFsL86G3EEYIvrtBTuaypdz+Cx7nd1iPZyWMx5Y1gXnVzha4nBdV4+7l9JIsFggD8QVpw2uHXQiS1KXFjOeqA3DBD8tjMB7q26Fl2fD3jkOo4BTbQ2NrRIZUu/iL+fOmMPsyMt2qulB0yaSBCfkbEq8xrUA="
file_glob: true
file:
- build/prod/cyberchef.htm
- build/node/CyberChef.js
- build/prod/*.zip
- src/node/cjs.js
on:
repo: gchq/CyberChef
tags: true
@ -36,7 +41,7 @@ deploy:
skip_cleanup: true
email: "n1474335@gmail.com"
api_key:
secure: "Z3FK6bm4RfQEIRXZ1lBNzQkVIoHpivThr9U+XBHmsBgIfdrK/XUnzs/slugo+NIz8nPiGmMx4gxyJonBCLHDGb1ysky2aEWTl26c0teaF4DeQEjWC1ZaGzv8MV1/GkUamnr1qouXjyUhyEAp33rd8ccN9Rq3QNYB/qLDcA9/FCme7JCW6sCd4zWO0LGEYMJEMc2FzAUkqhqsI05hegGhSDgKXRn5PmLARek4yHD+Hx7pstaTeQIy0WoGJjdzoB3iJIMmo/hWZGzZafktUOh223c5qzx4zMpDRNmMngBUw6R94nKd4KvplYRgB87Y3L/aiVU4CF+axwLmK8RPaC1wbJnlHf06zxHPdiFmsY/zKPpNel+nOnxzRrF5l2KMU4TU6gug3s9Jnzp9T5UMfhp0jW3YkxHGeuOPOeE1i0lTUWUGWrPHLQquAhLfkr2zxaU4ETk/y85hq9W4LAy0ENEDVXX2jP7FnI4Z1fdpmljpmVNJR+outPg6t+Coqgvil7v7XpMtDm8lKQanVYuxwmkb/ncOWFRWuM2j5zIEg3CHnFDcJ9bYrfKRg0b0tb/2BWD14pQnV76goVwzJQYVzdPc8TKIYJw2BZ1Nh9c0iruQVebe/6l1FX9fDCkz8VMmltni61/LxZrf8y0NT1YaU1raeNY2dH5UWvEa9p72FPMI6Eg="
secure: "UnDQL3Kh+GK2toL0TK3FObO0ujVssU3Eg4BBuYdjwLB81GhiGE5/DTh7THdZPOpbLo6wQeOwfZDuMeKC1OU+0Uf4NsdYFu1aq6xMO20qBQ4qUfgsyiK4Qgywj9gk0p1+OFZdGAZ/j1CNRAaF71XQIY6iV84c+SO4WoizXYrNT0Jh4sr2DA4/97G2xmJtPi0qOzYrJ09R56ZUozmqeik5G0pMRIuJRbpjS/7bZXV+N7WV0ombZc9RkUaetbabEVOLQ+Xx5YAIVq+VuEeMe9VBSnxY/FfCLmy1wJsjGzpLCyBI9nbrG4nw8Wgc2m8NfK9rcpIvBTGner9r2j60NVDkZ8kLZPrqXhq6AZMwa+oz6K5UQCqRo2RRQzSGwXxg67HY5Tcq+oNmjd+DqpPg4LZ3eGlluyP5XfG+hpSr9Ya4d8q8SrUWLxkoLHI6ZKMtoKFbTCSSQPiluW5hsZxjz3yDkkjsJw64M/EM8UyJrgaXqDklQu+7rBGKLfsK6os7RDiqjBWpQ7gwpo8HvY0O8yqEAabPz+QGkanpjcCOZCXFbSkzWxYy37RMAPu88iINVZVlZE4l+WJenCpZY95ueyy0mG9cyMSzVRPyX6A+/n4H6VMFPFjpGDLTD588ACEjY1lmHfS/eXwXJcgqPPD2gW0XdRdUheU/ssqlfCfGWQMTDXs="
on:
tags: true
branch: master

242
CHANGELOG.md
View file

@ -1,9 +1,143 @@
# Changelog
All notable changes to CyberChef will be documented in this file.
All major and minor version changes will be documented in this file. Details of patch-level version changes can be found in [commit messages](https://github.com/gchq/CyberChef/commits/master).
### [9.7.0] - 2019-09-13
- 'Optical Character Recognition' operation added [@MShwed] [@n1474335] | [#632]
### [9.6.0] - 2019-09-04
- 'Bacon Cipher Encode' and 'Bacon Cipher Decode' operations added [@kassi] | [#500]
### [9.5.0] - 2019-09-04
- Various Steganography operations added: 'Extract LSB', 'Extract RGBA', 'Randomize Colour Palette', and 'View Bit Plane' [@Ge0rg3] | [#625]
### [9.4.0] - 2019-08-30
- 'Render Markdown' operation added [@j433866] | [#627]
### [9.3.0] - 2019-08-30
- 'Show on map' operation added [@j433866] | [#477]
### [9.2.0] - 2019-08-23
- 'Parse UDP' operation added [@h345983745] | [#614]
### [9.1.0] - 2019-08-22
- 'Parse SSH Host Key' operation added [@j433866] | [#595]
- 'Defang IP Addresses' operation added [@h345983745] | [#556]
## [9.0.0] - 2019-07-09
- [Multiple inputs](https://github.com/gchq/CyberChef/wiki/Multiple-Inputs) are now supported in the main web UI, allowing you to upload and process multiple files at once [@j433866] | [#566]
- A [Node.js API](https://github.com/gchq/CyberChef/wiki/Node-API) has been implemented, meaning that CyberChef can now be used as a library, either to provide specific operations, or an entire baking environment [@d98762625] | [#291]
- A [read-eval-print loop (REPL)](https://github.com/gchq/CyberChef/wiki/Node-API#repl) is also included to enable prototyping and experimentation with the API [@d98762625] | [#291]
- Light and dark Solarized themes added [@j433866] | [#566]
<details>
<summary>Click to expand v8 minor versions</summary>
### [8.38.0] - 2019-07-03
- 'Streebog' and 'GOST hash' operations added [@MShwed] [@n1474335] | [#530]
### [8.37.0] - 2019-07-03
- 'CRC-8 Checksum' operation added [@MShwed] | [#591]
### [8.36.0] - 2019-07-03
- 'PGP Verify' operation added [@artemisbot] | [#585]
### [8.35.0] - 2019-07-03
- 'Sharpen Image', 'Convert Image Format' and 'Add Text To Image' operations added [@j433866] | [#515]
### [8.34.0] - 2019-06-28
- Various new visualisations added to the 'Entropy' operation [@MShwed] | [#535]
- Efficiency improvements made to the 'Entropy' operation for large file support [@n1474335]
### [8.33.0] - 2019-06-27
- 'Bzip2 Compress' operation added and 'Bzip2 Decompress' operation greatly improved [@artemisbot] | [#531]
### [8.32.0] - 2019-06-27
- 'Index of Coincidence' operation added [@Ge0rg3] | [#571]
### [8.31.0] - 2019-04-12
- The downloadable version of CyberChef is now a .zip file containing separate modules rather than a single .htm file. It is still completely standalone and will not make any external network requests. This change reduces the complexity of the build process significantly. [@n1474335]
### [8.30.0] - 2019-04-12
- 'Decode Protobuf' operation added [@n1474335] | [#533]
### [8.29.0] - 2019-03-31
- 'BLAKE2s' and 'BLAKE2b' hashing operations added [@h345983745] | [#525]
### [8.28.0] - 2019-03-31
- 'Heatmap Chart', 'Hex Density Chart', 'Scatter Chart' and 'Series Chart' operation added [@artemisbot] [@tlwr] | [#496] [#143]
### [8.27.0] - 2019-03-14
- 'Enigma', 'Typex', 'Bombe' and 'Multiple Bombe' operations added [@s2224834] | [#516]
- See [this wiki article](https://github.com/gchq/CyberChef/wiki/Enigma,-the-Bombe,-and-Typex) for a full explanation of these operations.
- New Bombe-style loading animation added for long-running operations [@n1474335]
- New operation argument types added: `populateMultiOption` and `argSelector` [@n1474335]
### [8.26.0] - 2019-03-09
- Various image manipulation operations added [@j433866] | [#506]
### [8.25.0] - 2019-03-09
- 'Extract Files' operation added and more file formats supported [@n1474335] | [#440]
### [8.24.0] - 2019-02-08
- 'DNS over HTTPS' operation added [@h345983745] | [#489]
### [8.23.1] - 2019-01-18
- 'Convert co-ordinate format' operation added [@j433866] | [#476]
### [8.23.0] - 2019-01-18
- 'YARA Rules' operation added [@artemisbot] | [#468]
### [8.22.0] - 2019-01-10
- 'Subsection' operation added [@j433866] | [#467]
### [8.21.0] - 2019-01-10
- 'To Case Insensitive Regex' and 'From Case Insensitive Regex' operations added [@masq] | [#461]
### [8.20.0] - 2019-01-09
- 'Generate Lorem Ipsum' operation added [@klaxon1] | [#455]
### [8.19.0] - 2018-12-30
- UI test suite added to confirm that the app loads correctly in a reasonable time and that various operations from each module can be run [@n1474335] | [#458]
### [8.18.0] - 2018-12-26
- 'Split Colour Channels' operation added [@artemisbot] | [#449]
### [8.17.0] - 2018-12-25
- 'Generate QR Code' and 'Parse QR Code' operations added [@j433866] | [#448]
### [8.16.0] - 2018-12-19
- 'Play Media' operation added [@anthony-arnold] | [#446]
### [8.15.0] - 2018-12-18
- 'Text Encoding Brute Force' operation added [@Cynser] | [#439]
### [8.14.0] - 2018-12-18
- 'To Base62' and 'From Base62' operations added [@tcode2k16] | [#443]
### [8.13.0] - 2018-12-15
- 'A1Z26 Cipher Encode' and 'A1Z26 Cipher Decode' operations added [@jarmovanlenthe] | [#441]
### [8.12.0] - 2018-11-21
- 'Citrix CTX1 Encode' and 'Citrix CTX1 Decode' operations added [@bwhitn] | [#428]
### [8.11.0] - 2018-11-13
- 'CSV to JSON' and 'JSON to CSV' operations added [@n1474335] | [#277]
### [8.10.0] - 2018-11-07
- 'Remove Diacritics' operation added [@klaxon1] | [#387]
### [8.9.0] - 2018-11-07
- 'Defang URL' operation added [@arnydo] | [#394]
### [8.8.0] - 2018-10-10
- 'Parse TLV' operation added [@GCHQ77703] | [#351]
### [8.7.0] - 2018-08-31
- 'JWT Sign', 'JWT Verify' and 'JWT Decode' operations added [@GCHQ77703] | [#348]
### [8.6.0] - 2018-08-29
- 'To Geohash' and 'From Geohash' operations added [@GCHQ77703] | [#344]
- 'To Geohash' and 'From Geohash' operations added [@GCHQ77703] | [#344]
### [8.5.0] - 2018-08-23
- 'To Braille' and 'From Braille' operations added [@n1474335] | [#255]
@ -20,6 +154,8 @@ All notable changes to CyberChef will be documented in this file.
### [8.1.0] - 2018-08-19
- 'Dechunk HTTP response' operation added [@sevzero] | [#311]
</details>
## [8.0.0] - 2018-08-05
- Codebase rewritten using [ES modules](https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/) and [classes](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes) [@n1474335] [@d98762625] [@artemisbot] [@picapi] | [#284]
- Operation architecture restructured to make adding new operations a lot simpler [@n1474335] | [#284]
@ -38,7 +174,7 @@ All notable changes to CyberChef will be documented in this file.
- Added support for loading, processing and downloading files up to 500MB [@n1474335] | [#224]
## [6.0.0] - 2017-09-19
- Threading support added. All recipe processing moved into a [Web Worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers) to increase performance and allowing long-running operations to be cancelled [@n1474335] | [#173]
- Threading support added. All recipe processing moved into a [Web Worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers) to increase performance and to allow long-running operations to be cancelled [@n1474335] | [#173]
- Module system created so that operations relying on large libraries can be downloaded separately as required, reducing the initial loading time for the app [@n1474335] | [#173]
## [5.0.0] - 2017-03-30
@ -48,6 +184,48 @@ All notable changes to CyberChef will be documented in this file.
- Initial open source commit [@n1474335] | [b1d73a72](https://github.com/gchq/CyberChef/commit/b1d73a725dc7ab9fb7eb789296efd2b7e4b08306)
[9.7.0]: https://github.com/gchq/CyberChef/releases/tag/v9.7.0
[9.6.0]: https://github.com/gchq/CyberChef/releases/tag/v9.6.0
[9.5.0]: https://github.com/gchq/CyberChef/releases/tag/v9.5.0
[9.4.0]: https://github.com/gchq/CyberChef/releases/tag/v9.4.0
[9.3.0]: https://github.com/gchq/CyberChef/releases/tag/v9.3.0
[9.2.0]: https://github.com/gchq/CyberChef/releases/tag/v9.2.0
[9.1.0]: https://github.com/gchq/CyberChef/releases/tag/v9.1.0
[9.0.0]: https://github.com/gchq/CyberChef/releases/tag/v9.0.0
[8.38.0]: https://github.com/gchq/CyberChef/releases/tag/v8.38.0
[8.37.0]: https://github.com/gchq/CyberChef/releases/tag/v8.37.0
[8.36.0]: https://github.com/gchq/CyberChef/releases/tag/v8.36.0
[8.35.0]: https://github.com/gchq/CyberChef/releases/tag/v8.35.0
[8.34.0]: https://github.com/gchq/CyberChef/releases/tag/v8.34.0
[8.33.0]: https://github.com/gchq/CyberChef/releases/tag/v8.33.0
[8.32.0]: https://github.com/gchq/CyberChef/releases/tag/v8.32.0
[8.31.0]: https://github.com/gchq/CyberChef/releases/tag/v8.31.0
[8.30.0]: https://github.com/gchq/CyberChef/releases/tag/v8.30.0
[8.29.0]: https://github.com/gchq/CyberChef/releases/tag/v8.29.0
[8.28.0]: https://github.com/gchq/CyberChef/releases/tag/v8.28.0
[8.27.0]: https://github.com/gchq/CyberChef/releases/tag/v8.27.0
[8.26.0]: https://github.com/gchq/CyberChef/releases/tag/v8.26.0
[8.25.0]: https://github.com/gchq/CyberChef/releases/tag/v8.25.0
[8.24.0]: https://github.com/gchq/CyberChef/releases/tag/v8.24.0
[8.23.1]: https://github.com/gchq/CyberChef/releases/tag/v8.23.1
[8.23.0]: https://github.com/gchq/CyberChef/releases/tag/v8.23.0
[8.22.0]: https://github.com/gchq/CyberChef/releases/tag/v8.22.0
[8.21.0]: https://github.com/gchq/CyberChef/releases/tag/v8.21.0
[8.20.0]: https://github.com/gchq/CyberChef/releases/tag/v8.20.0
[8.19.0]: https://github.com/gchq/CyberChef/releases/tag/v8.19.0
[8.18.0]: https://github.com/gchq/CyberChef/releases/tag/v8.18.0
[8.17.0]: https://github.com/gchq/CyberChef/releases/tag/v8.17.0
[8.16.0]: https://github.com/gchq/CyberChef/releases/tag/v8.16.0
[8.15.0]: https://github.com/gchq/CyberChef/releases/tag/v8.15.0
[8.14.0]: https://github.com/gchq/CyberChef/releases/tag/v8.14.0
[8.13.0]: https://github.com/gchq/CyberChef/releases/tag/v8.13.0
[8.12.0]: https://github.com/gchq/CyberChef/releases/tag/v8.12.0
[8.11.0]: https://github.com/gchq/CyberChef/releases/tag/v8.11.0
[8.10.0]: https://github.com/gchq/CyberChef/releases/tag/v8.10.0
[8.9.0]: https://github.com/gchq/CyberChef/releases/tag/v8.9.0
[8.8.0]: https://github.com/gchq/CyberChef/releases/tag/v8.8.0
[8.7.0]: https://github.com/gchq/CyberChef/releases/tag/v8.7.0
[8.6.0]: https://github.com/gchq/CyberChef/releases/tag/v8.6.0
[8.5.0]: https://github.com/gchq/CyberChef/releases/tag/v8.5.0
[8.4.0]: https://github.com/gchq/CyberChef/releases/tag/v8.4.0
@ -62,22 +240,40 @@ All notable changes to CyberChef will be documented in this file.
[@n1474335]: https://github.com/n1474335
[@d98762625]: https://github.com/d98762625
[@j433866]: https://github.com/j433866
[@GCHQ77703]: https://github.com/GCHQ77703
[@h345983745]: https://github.com/h345983745
[@s2224834]: https://github.com/s2224834
[@artemisbot]: https://github.com/artemisbot
[@tlwr]: https://github.com/tlwr
[@picapi]: https://github.com/picapi
[@Dachande663]: https://github.com/Dachande663
[@JustAnotherMark]: https://github.com/JustAnotherMark
[@sevzero]: https://github.com/sevzero
[@PenguinGeorge]: https://github.com/PenguinGeorge
[@arnydo]: https://github.com/arnydo
[@klaxon1]: https://github.com/klaxon1
[@bwhitn]: https://github.com/bwhitn
[@jarmovanlenthe]: https://github.com/jarmovanlenthe
[@tcode2k16]: https://github.com/tcode2k16
[@Cynser]: https://github.com/Cynser
[@anthony-arnold]: https://github.com/anthony-arnold
[@masq]: https://github.com/masq
[@Ge0rg3]: https://github.com/Ge0rg3
[@MShwed]: https://github.com/MShwed
[@kassi]: https://github.com/kassi
[#95]: https://github.com/gchq/CyberChef/pull/299
[#173]: https://github.com/gchq/CyberChef/pull/173
[#143]: https://github.com/gchq/CyberChef/pull/143
[#224]: https://github.com/gchq/CyberChef/pull/224
[#239]: https://github.com/gchq/CyberChef/pull/239
[#248]: https://github.com/gchq/CyberChef/pull/248
[#255]: https://github.com/gchq/CyberChef/issues/255
[#277]: https://github.com/gchq/CyberChef/issues/277
[#281]: https://github.com/gchq/CyberChef/pull/281
[#284]: https://github.com/gchq/CyberChef/pull/284
[#291]: https://github.com/gchq/CyberChef/pull/291
[#294]: https://github.com/gchq/CyberChef/pull/294
[#296]: https://github.com/gchq/CyberChef/pull/296
[#298]: https://github.com/gchq/CyberChef/pull/298
@ -86,3 +282,43 @@ All notable changes to CyberChef will be documented in this file.
[#338]: https://github.com/gchq/CyberChef/pull/338
[#340]: https://github.com/gchq/CyberChef/pull/340
[#344]: https://github.com/gchq/CyberChef/pull/344
[#348]: https://github.com/gchq/CyberChef/pull/348
[#351]: https://github.com/gchq/CyberChef/pull/351
[#387]: https://github.com/gchq/CyberChef/pull/387
[#394]: https://github.com/gchq/CyberChef/pull/394
[#428]: https://github.com/gchq/CyberChef/pull/428
[#439]: https://github.com/gchq/CyberChef/pull/439
[#440]: https://github.com/gchq/CyberChef/pull/440
[#441]: https://github.com/gchq/CyberChef/pull/441
[#443]: https://github.com/gchq/CyberChef/pull/443
[#446]: https://github.com/gchq/CyberChef/pull/446
[#448]: https://github.com/gchq/CyberChef/pull/448
[#449]: https://github.com/gchq/CyberChef/pull/449
[#455]: https://github.com/gchq/CyberChef/pull/455
[#458]: https://github.com/gchq/CyberChef/pull/458
[#461]: https://github.com/gchq/CyberChef/pull/461
[#467]: https://github.com/gchq/CyberChef/pull/467
[#468]: https://github.com/gchq/CyberChef/pull/468
[#476]: https://github.com/gchq/CyberChef/pull/476
[#477]: https://github.com/gchq/CyberChef/pull/477
[#489]: https://github.com/gchq/CyberChef/pull/489
[#496]: https://github.com/gchq/CyberChef/pull/496
[#500]: https://github.com/gchq/CyberChef/pull/500
[#506]: https://github.com/gchq/CyberChef/pull/506
[#515]: https://github.com/gchq/CyberChef/pull/515
[#516]: https://github.com/gchq/CyberChef/pull/516
[#525]: https://github.com/gchq/CyberChef/pull/525
[#530]: https://github.com/gchq/CyberChef/pull/530
[#531]: https://github.com/gchq/CyberChef/pull/531
[#533]: https://github.com/gchq/CyberChef/pull/533
[#535]: https://github.com/gchq/CyberChef/pull/535
[#556]: https://github.com/gchq/CyberChef/pull/556
[#566]: https://github.com/gchq/CyberChef/pull/566
[#571]: https://github.com/gchq/CyberChef/pull/571
[#585]: https://github.com/gchq/CyberChef/pull/585
[#591]: https://github.com/gchq/CyberChef/pull/591
[#595]: https://github.com/gchq/CyberChef/pull/595
[#614]: https://github.com/gchq/CyberChef/pull/614
[#625]: https://github.com/gchq/CyberChef/pull/625
[#627]: https://github.com/gchq/CyberChef/pull/627
[#632]: https://github.com/gchq/CyberChef/pull/632

378
Gruntfile.js
View file

@ -2,8 +2,7 @@
const webpack = require("webpack");
const HtmlWebpackPlugin = require("html-webpack-plugin");
const NodeExternals = require("webpack-node-externals");
const Inliner = require("web-resource-inliner");
const BundleAnalyzerPlugin = require("webpack-bundle-analyzer").BundleAnalyzerPlugin;
const glob = require("glob");
const path = require("path");
@ -15,6 +14,7 @@ const path = require("path");
* @license Apache-2.0
*/
module.exports = function (grunt) {
grunt.file.defaultEncoding = "utf8";
grunt.file.preserveBOM = false;
@ -24,33 +24,38 @@ module.exports = function (grunt) {
"A persistent task which creates a development build whenever source files are modified.",
["clean:dev", "clean:config", "exec:generateConfig", "concurrent:dev"]);
grunt.registerTask("node",
"Compiles CyberChef into a single NodeJS module.",
["clean:node", "clean:config", "exec:generateConfig", "webpack:node", "chmod:build"]);
grunt.registerTask("test",
"A task which runs all the tests in test/tests.",
["exec:generateConfig", "exec:tests"]);
grunt.registerTask("docs",
"Compiles documentation in the /docs directory.",
["clean:docs", "jsdoc", "chmod:docs"]);
grunt.registerTask("prod",
"Creates a production-ready build. Use the --msg flag to add a compile message.",
["eslint", "clean:prod", "clean:config", "exec:generateConfig", "webpack:web", "inline", "chmod"]);
[
"eslint", "clean:prod", "clean:config", "exec:generateConfig", "webpack:web",
"copy:standalone", "zip:standalone", "clean:standalone", "chmod"
]);
grunt.registerTask("node",
"Compiles CyberChef into a single NodeJS module.",
[
"clean:node", "clean:config", "clean:nodeConfig", "exec:generateConfig", "exec:generateNodeIndex"
]);
grunt.registerTask("test",
"A task which runs all the operation tests in the tests directory.",
[
"clean:config", "clean:nodeConfig", "exec:generateConfig", "exec:generateNodeIndex",
"exec:nodeTests", "exec:opTests"
]);
grunt.registerTask("testui",
"A task which runs all the UI tests in the tests directory. The prod task must already have been run.",
["connect:prod", "exec:browserTests"]);
grunt.registerTask("testnodeconsumer",
"A task which checks whether consuming CJS and ESM apps work with the CyberChef build",
["exec:setupNodeConsumers", "exec:testCJSNodeConsumer", "exec:testESMNodeConsumer", "exec:testESMDeepImportNodeConsumer", "exec:teardownNodeConsumers"]);
grunt.registerTask("default",
"Lints the code base",
["eslint", "exec:repoSize"]);
grunt.registerTask("inline",
"Compiles a production build of CyberChef into a single, portable web page.",
["exec:generateConfig", "webpack:webInline", "runInliner", "clean:inlineScripts"]);
grunt.registerTask("runInliner", runInliner);
grunt.registerTask("doc", "docs");
grunt.registerTask("tests", "test");
grunt.registerTask("lint", "eslint");
@ -58,7 +63,6 @@ module.exports = function (grunt) {
// Load tasks provided by each plugin
grunt.loadNpmTasks("grunt-eslint");
grunt.loadNpmTasks("grunt-webpack");
grunt.loadNpmTasks("grunt-jsdoc");
grunt.loadNpmTasks("grunt-contrib-clean");
grunt.loadNpmTasks("grunt-contrib-copy");
grunt.loadNpmTasks("grunt-contrib-watch");
@ -66,6 +70,8 @@ module.exports = function (grunt) {
grunt.loadNpmTasks("grunt-exec");
grunt.loadNpmTasks("grunt-accessibility");
grunt.loadNpmTasks("grunt-concurrent");
grunt.loadNpmTasks("grunt-contrib-connect");
grunt.loadNpmTasks("grunt-zip");
// Project configuration
@ -76,44 +82,10 @@ module.exports = function (grunt) {
COMPILE_TIME: JSON.stringify(compileTime),
COMPILE_MSG: JSON.stringify(grunt.option("compile-msg") || grunt.option("msg") || ""),
PKG_VERSION: JSON.stringify(pkg.version),
ENVIRONMENT_IS_WORKER: function() {
return typeof importScripts === "function";
},
ENVIRONMENT_IS_NODE: function() {
return typeof process === "object" && typeof require === "function";
},
ENVIRONMENT_IS_WEB: function() {
return typeof window === "object";
}
},
moduleEntryPoints = listEntryModules();
moduleEntryPoints = listEntryModules(),
nodeConsumerTestPath = "~/tmp-cyberchef";
/**
* Compiles a production build of CyberChef into a single, portable web page.
*/
function runInliner() {
const done = this.async();
Inliner.html({
relativeTo: "build/prod/",
fileContent: grunt.file.read("build/prod/cyberchef.htm"),
images: true,
svgs: true,
scripts: true,
links: true,
strict: true
}, function(error, result) {
if (error) {
if (error instanceof Error) {
done(error);
} else {
done(new Error(error));
}
} else {
grunt.file.write("build/prod/cyberchef.htm", result);
done(true);
}
});
}
/**
* Generates an entry list for all the modules.
@ -124,7 +96,7 @@ module.exports = function (grunt) {
glob.sync("./src/core/config/modules/*.mjs").forEach(file => {
const basename = path.basename(file);
if (basename !== "Default.mjs" && basename !== "OpModules.mjs")
entryModules[basename.split(".mjs")[0]] = path.resolve(file);
entryModules["modules/" + basename.split(".mjs")[0]] = path.resolve(file);
});
return entryModules;
@ -136,34 +108,18 @@ module.exports = function (grunt) {
prod: ["build/prod/*"],
node: ["build/node/*"],
config: ["src/core/config/OperationConfig.json", "src/core/config/modules/*", "src/code/operations/index.mjs"],
docs: ["docs/*", "!docs/*.conf.json", "!docs/*.ico", "!docs/*.png"],
inlineScripts: ["build/prod/scripts.js"],
nodeConfig: ["src/node/index.mjs", "src/node/config/OperationConfig.json"],
standalone: ["build/prod/CyberChef*.html"]
},
eslint: {
options: {
configFile: "./.eslintrc.json"
},
configs: ["Gruntfile.js"],
configs: ["*.{js,mjs}"],
core: ["src/core/**/*.{js,mjs}", "!src/core/vendor/**/*", "!src/core/operations/legacy/**/*"],
web: ["src/web/**/*.{js,mjs}"],
web: ["src/web/**/*.{js,mjs}", "!src/web/static/**/*"],
node: ["src/node/**/*.{js,mjs}"],
tests: ["test/**/*.{js,mjs}"],
},
jsdoc: {
options: {
destination: "docs",
template: "node_modules/ink-docstrap/template",
recurse: true,
readme: "./README.md",
configure: "docs/jsdoc.conf.json"
},
all: {
src: [
"src/**/*.js",
"src/**/*.mjs",
"!src/core/vendor/**/*"
],
}
tests: ["tests/**/*.{js,mjs}"],
},
accessibility: {
options: {
@ -179,93 +135,48 @@ module.exports = function (grunt) {
},
webpack: {
options: webpackConfig,
web: {
mode: "production",
target: "web",
entry: Object.assign({
main: "./src/web/index.js",
sitemap: "./src/web/static/sitemap.js"
}, moduleEntryPoints),
output: {
path: __dirname + "/build/prod"
},
resolve: {
alias: {
"./config/modules/OpModules": "./config/modules/Default"
}
},
plugins: [
new webpack.DefinePlugin(BUILD_CONSTANTS),
new HtmlWebpackPlugin({
filename: "index.html",
template: "./src/web/html/index.html",
chunks: ["main"],
compileTime: compileTime,
version: pkg.version,
minify: {
removeComments: true,
collapseWhitespace: true,
minifyJS: true,
minifyCSS: true
web: () => {
return {
mode: "production",
target: "web",
entry: Object.assign({
main: "./src/web/index.js"
}, moduleEntryPoints),
output: {
path: __dirname + "/build/prod",
filename: chunkData => {
return chunkData.chunk.name === "main" ? "assets/[name].js": "[name].js";
},
globalObject: "this"
},
resolve: {
alias: {
"./config/modules/OpModules.mjs": "./config/modules/Default.mjs"
}
}),
]
},
plugins: [
new webpack.DefinePlugin(BUILD_CONSTANTS),
new HtmlWebpackPlugin({
filename: "index.html",
template: "./src/web/html/index.html",
chunks: ["main"],
compileTime: compileTime,
version: pkg.version,
minify: {
removeComments: true,
collapseWhitespace: true,
minifyJS: true,
minifyCSS: true
}
}),
new BundleAnalyzerPlugin({
analyzerMode: "static",
reportFilename: "BundleAnalyzerReport.html",
openAnalyzer: false
}),
]
};
},
webInline: {
mode: "production",
target: "web",
entry: "./src/web/index.js",
output: {
filename: "scripts.js",
path: __dirname + "/build/prod"
},
plugins: [
new webpack.DefinePlugin(Object.assign({}, BUILD_CONSTANTS, {
INLINE: "true"
})),
new HtmlWebpackPlugin({
filename: "cyberchef.htm",
template: "./src/web/html/index.html",
compileTime: compileTime,
version: pkg.version + "s",
inline: true,
minify: {
removeComments: true,
collapseWhitespace: true,
minifyJS: true,
minifyCSS: true
}
}),
]
},
tests: {
mode: "development",
target: "node",
entry: "./test/index.mjs",
externals: [NodeExternals()],
output: {
filename: "index.js",
path: __dirname + "/build/test"
},
plugins: [
new webpack.DefinePlugin(BUILD_CONSTANTS)
]
},
node: {
mode: "production",
target: "node",
entry: "./src/node/index.mjs",
externals: [NodeExternals()],
output: {
filename: "CyberChef.js",
path: __dirname + "/build/node",
library: "CyberChef",
libraryTarget: "commonjs2"
},
plugins: [
new webpack.DefinePlugin(BUILD_CONSTANTS)
]
}
},
"webpack-dev-server": {
options: {
@ -280,7 +191,12 @@ module.exports = function (grunt) {
chunks: false,
modules: false,
entrypoints: false,
warningsFilter: [/source-map/, /dependency is an expression/],
warningsFilter: [
/source-map/,
/dependency is an expression/,
/export 'default'/,
/Can't resolve 'sodium'/
],
}
},
start: {
@ -292,7 +208,7 @@ module.exports = function (grunt) {
}, moduleEntryPoints),
resolve: {
alias: {
"./config/modules/OpModules": "./config/modules/Default"
"./config/modules/OpModules.mjs": "./config/modules/Default.mjs"
}
},
plugins: [
@ -308,14 +224,39 @@ module.exports = function (grunt) {
}
}
},
zip: {
standalone: {
cwd: "build/prod/",
src: [
"build/prod/**/*",
"!build/prod/index.html",
"!build/prod/BundleAnalyzerReport.html",
],
dest: `build/prod/CyberChef_v${pkg.version}.zip`
}
},
connect: {
prod: {
options: {
port: 8000,
base: "build/prod/"
}
}
},
copy: {
ghPages: {
options: {
process: function (content, srcpath) {
// Add Google Analytics code to index.html
if (srcpath.indexOf("index.html") >= 0) {
// Add Google Analytics code to index.html
content = content.replace("</body></html>",
grunt.file.read("src/web/static/ga.html") + "</body></html>");
// Add Structured Data for SEO
content = content.replace("</head>",
"<script type='application/ld+json'>" +
JSON.stringify(JSON.parse(grunt.file.read("src/web/static/structuredData.json"))) +
"</script></head>");
return grunt.template.process(content, srcpath);
} else {
return content;
@ -327,12 +268,29 @@ module.exports = function (grunt) {
{
src: "build/prod/index.html",
dest: "build/prod/index.html"
}
]
},
standalone: {
options: {
process: function (content, srcpath) {
if (srcpath.indexOf("index.html") >= 0) {
// Replace download link with version number
content = content.replace(/<a [^>]+>Download CyberChef.+?<\/a>/,
`<span>Version ${pkg.version}</span>`);
return grunt.template.process(content, srcpath);
} else {
return content;
}
},
noProcess: ["**", "!**/*.html"]
},
files: [
{
expand: true,
src: "docs/**",
dest: "build/prod/"
},
src: "build/prod/index.html",
dest: `build/prod/CyberChef_v${pkg.version}.html`
}
]
}
},
@ -342,18 +300,12 @@ module.exports = function (grunt) {
mode: "755",
},
src: ["build/**/*", "build/"]
},
docs: {
options: {
mode: "755",
},
src: ["docs/**/*", "docs/"]
}
},
watch: {
config: {
files: ["src/core/operations/**/*", "!src/core/operations/index.mjs"],
tasks: ["exec:generateConfig"]
tasks: ["exec:generateNodeIndex", "exec:generateConfig"]
}
},
concurrent: {
@ -374,22 +326,70 @@ module.exports = function (grunt) {
command: "git gc --prune=now --aggressive"
},
sitemap: {
command: "node build/prod/sitemap.js > build/prod/sitemap.xml"
command: "node --experimental-modules --no-warnings --no-deprecation src/web/static/sitemap.mjs > build/prod/sitemap.xml"
},
generateConfig: {
command: [
"echo '\n--- Regenerating config files. ---'",
"mkdir -p src/core/config/modules",
"echo 'export default {};\n' > src/core/config/modules/OpModules.mjs",
"echo '[]\n' > src/core/config/OperationConfig.json",
"node --experimental-modules src/core/config/scripts/generateOpsIndex.mjs",
"node --experimental-modules src/core/config/scripts/generateConfig.mjs",
"echo [] > src/core/config/OperationConfig.json",
"node --experimental-modules --no-warnings --no-deprecation src/core/config/scripts/generateOpsIndex.mjs",
"node --experimental-modules --no-warnings --no-deprecation src/core/config/scripts/generateConfig.mjs",
"echo '--- Config scripts finished. ---\n'"
].join(";")
},
tests: {
command: "node --experimental-modules test/index.mjs"
}
generateNodeIndex: {
command: [
"echo '\n--- Regenerating node index ---'",
"node --experimental-modules --no-warnings --no-deprecation src/node/config/scripts/generateNodeIndex.mjs",
"echo '--- Node index generated. ---\n'"
].join(";"),
},
opTests: {
command: "node --experimental-modules --no-warnings --no-deprecation tests/operations/index.mjs"
},
browserTests: {
command: "./node_modules/.bin/nightwatch --env prod"
},
nodeTests: {
command: "node --experimental-modules --no-warnings --no-deprecation tests/node/index.mjs"
},
setupNodeConsumers: {
command: [
"echo '\n--- Testing node conumers ---'",
"npm link",
`mkdir ${nodeConsumerTestPath}`,
`cp tests/node/consumers/* ${nodeConsumerTestPath}`,
`cd ${nodeConsumerTestPath}`,
"npm link cyberchef"
].join(";"),
},
teardownNodeConsumers: {
command: [
`rm -rf ${nodeConsumerTestPath}`,
"echo '\n--- Node consumer tests complete ---'"
].join(";"),
},
testCJSNodeConsumer: {
command: [
`cd ${nodeConsumerTestPath}`,
"node --no-warnings cjs-consumer.js",
].join(";"),
stdout: false,
},
testESMNodeConsumer: {
command: [
`cd ${nodeConsumerTestPath}`,
"node --no-warnings --experimental-modules esm-consumer.mjs",
].join(";"),
stdout: false,
},
testESMDeepImportNodeConsumer: {
command: [
`cd ${nodeConsumerTestPath}`,
"node --no-warnings --experimental-modules esm-deep-import-consumer.mjs",
].join(";"),
stdout: false,
},
},
});
};

15
README.md
View file

@ -3,7 +3,6 @@
[![Build Status](https://travis-ci.org/gchq/CyberChef.svg?branch=master)](https://travis-ci.org/gchq/CyberChef)
[![dependencies Status](https://david-dm.org/gchq/CyberChef/status.svg)](https://david-dm.org/gchq/CyberChef)
[![npm](https://img.shields.io/npm/v/cyberchef.svg)](https://www.npmjs.com/package/cyberchef)
![](https://reposs.herokuapp.com/?path=gchq/CyberChef&color=blue)
[![](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/gchq/CyberChef/blob/master/LICENSE)
[![Gitter](https://badges.gitter.im/gchq/CyberChef.svg)](https://gitter.im/gchq/CyberChef?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
@ -50,7 +49,7 @@ You can use as many operations as you like in simple or complex ways. Some examp
- Drag and drop
- Operations can be dragged in and out of the recipe list, or reorganised.
- Files up to 500MB can be dragged over the input box to load them directly into the browser.
- Files up to 2GB can be dragged over the input box to load them directly into the browser.
- Auto Bake
- Whenever you modify the input or the recipe, CyberChef will automatically "bake" for you and produce the output immediately.
- This can be turned off and operated manually if it is affecting performance (if the input is very large, for instance).
@ -67,7 +66,7 @@ You can use as many operations as you like in simple or complex ways. Some examp
- Highlighting
- When you highlight text in the input or output, the offset and length values will be displayed and, if possible, the corresponding data will be highlighted in the output or input respectively (example: [highlight the word 'question' in the input to see where it appears in the output][11]).
- Save to file and load from file
- You can save the output to a file at any time or load a file by dragging and dropping it into the input field. Files up to around 500MB are supported (depending on your browser), however some operations may take a very long time to run over this much data.
- You can save the output to a file at any time or load a file by dragging and dropping it into the input field. Files up to around 2GB are supported (depending on your browser), however some operations may take a very long time to run over this much data.
- CyberChef is entirely client-side
- It should be noted that none of your recipe configuration or input (either text or files) is ever sent to the CyberChef web server - all processing is carried out within your browser, on your own computer.
- Due to this feature, CyberChef can be compiled into a single HTML file. You can download this file and drop it into a virtual machine, share it with other people, or use it independently on your local machine.
@ -77,9 +76,13 @@ You can use as many operations as you like in simple or complex ways. Some examp
CyberChef is built to support
- Google Chrome 40+
- Mozilla Firefox 35+
- Microsoft Edge 14+
- Google Chrome 50+
- Mozilla Firefox 38+
## Node.js support
CyberChef is built to fully support Node.js `v10` and partially supports `v12`. Named imports using a deep import specifier does not work in `v12`. For more information, see the Node API page in the project [wiki pages](https://github.com/gchq/CyberChef/wiki)
## Contributing

26
SECURITY.md Normal file
View file

@ -0,0 +1,26 @@
# Security Policy
## Supported Versions
CyberChef is supported on a best endeavours basis. Patches will be applied to
the latest version rather than retroactively to older versions. To ensure you
are using the most secure version of CyberChef, please make sure you have the
[latest release](https://github.com/gchq/CyberChef/releases/latest). The
official [live demo](https://gchq.github.io/CyberChef/) is always up to date.
## Reporting a Vulnerability
In most scenarios, the most appropriate way to report a vulnerability is to
[raise a new issue](https://github.com/gchq/CyberChef/issues/new/choose)
describing the problem in as much detail as possible, ideally with examples.
This will obviously be public. If you feel that the vulnerability is
significant enough to warrant a private disclosure, please email
[oss@gchq.gov.uk](mailto:oss@gchq.gov.uk) and
[n1474335@gmail.com](mailto:n1474335@gmail.com).
Disclosures of vulnerabilities in CyberChef are always welcomed. Whilst we aim
to write clean and secure code free from bugs, we recognise that this is an open
source project written by analysts in their spare time, relying on dozens of
open source libraries that are modified and updated on a regular basis. We hope
that the community will continue to support us as we endeavour to maintain and
develop this tool together.

26
babel.config.js Normal file
View file

@ -0,0 +1,26 @@
module.exports = function(api) {
api.cache.forever();
return {
"presets": [
["@babel/preset-env", {
"modules": false,
"useBuiltIns": "entry",
"corejs": 3
}]
],
"plugins": [
"dynamic-import-node",
[
"babel-plugin-transform-builtin-extend", {
"globals": ["Error"]
}
],
[
"@babel/plugin-transform-runtime", {
"regenerator": true
}
]
]
};
};

BIN
docs/favicon.ico
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 1.1 KiB

32
docs/jsdoc.conf.json
View file

@ -1,32 +0,0 @@
{
"tags": {
"allowUnknownTags": true
},
"plugins": [
"plugins/markdown",
"node_modules/jsdoc-babel"
],
"templates": {
"systemName": "CyberChef",
"footer": "",
"copyright": "&copy; Crown Copyright 2017",
"navType": "inline",
"theme": "cerulean",
"linenums": true,
"collapseSymbols": false,
"inverseNav": true,
"outputSourceFiles": true,
"outputSourcePath": true,
"dateFormat": "ddd MMM Do YYYY",
"sort": false,
"logoFile": "cyberchef-32x32.png",
"cleverLinks": false,
"monospaceLinks": false,
"protocol": "html://",
"methodHeadingReturns": false
},
"markdown": {
"parser": "gfm",
"hardwrap": true
}
}

30
nightwatch.json Normal file
View file

@ -0,0 +1,30 @@
{
"src_folders": ["tests/browser"],
"output_folder": "tests/browser/output",
"test_settings": {
"default": {
"launch_url": "http://localhost:8080",
"webdriver": {
"start_process": true,
"server_path": "./node_modules/.bin/chromedriver",
"port": 9515,
"log_path": false
},
"desiredCapabilities": {
"browserName": "chrome"
}
},
"dev": {
"launch_url": "http://localhost:8080"
},
"prod": {
"launch_url": "http://localhost:8000/index.html"
}
}
}

10739
package-lock.json generated
View file

File diff suppressed because it is too large Load diff

160
package.json
View file

@ -1,6 +1,6 @@
{
"name": "cyberchef",
"version": "8.6.2",
"version": "9.7.2",
"description": "The Cyber Swiss Army Knife for encryption, encoding, compression and data analysis.",
"author": "n1474335 <n1474335@gmail.com>",
"homepage": "https://gchq.github.io/CyberChef",
@ -27,113 +27,141 @@
"type": "git",
"url": "https://github.com/gchq/CyberChef/"
},
"main": "build/node/CyberChef.js",
"main": "src/node/cjs.js",
"module": "src/node/index.mjs",
"bugs": "https://github.com/gchq/CyberChef/issues",
"browserslist": [
"Chrome >= 50",
"Firefox >= 38",
"node >= 10"
],
"devDependencies": {
"autoprefixer": "^9.1.0",
"babel-core": "^6.26.3",
"babel-loader": "^7.1.5",
"babel-preset-env": "^1.7.0",
"bootstrap": "^4.1.3",
"colors": "^1.3.1",
"css-loader": "^1.0.0",
"eslint": "^5.3.0",
"@babel/core": "^7.5.5",
"@babel/plugin-transform-runtime": "^7.5.5",
"@babel/preset-env": "^7.5.5",
"autoprefixer": "^9.6.1",
"babel-eslint": "^10.0.3",
"babel-loader": "^8.0.6",
"babel-plugin-dynamic-import-node": "^2.3.0",
"chromedriver": "^76.0.1",
"colors": "^1.3.3",
"copy-webpack-plugin": "^5.0.4",
"css-loader": "^3.2.0",
"eslint": "^6.2.2",
"exports-loader": "^0.7.0",
"extract-text-webpack-plugin": "^4.0.0-alpha0",
"file-loader": "^1.1.11",
"grunt": "^1.0.3",
"file-loader": "^4.2.0",
"grunt": "^1.0.4",
"grunt-accessibility": "~6.0.0",
"grunt-chmod": "~1.1.1",
"grunt-concurrent": "^2.3.1",
"grunt-contrib-clean": "~1.1.0",
"grunt-concurrent": "^3.0.0",
"grunt-contrib-clean": "~2.0.0",
"grunt-contrib-connect": "^2.0.0",
"grunt-contrib-copy": "~1.0.0",
"grunt-contrib-watch": "^1.1.0",
"grunt-eslint": "^21.0.0",
"grunt-eslint": "^22.0.0",
"grunt-exec": "~3.0.0",
"grunt-jsdoc": "^2.2.1",
"grunt-webpack": "^3.1.2",
"grunt-webpack": "^3.1.3",
"grunt-zip": "^0.18.2",
"html-webpack-plugin": "^3.2.0",
"imports-loader": "^0.8.0",
"ink-docstrap": "^1.3.2",
"js-to-mjs": "^0.2.0",
"jsdoc-babel": "^0.4.0",
"node-sass": "^4.9.2",
"postcss-css-variables": "^0.9.0",
"postcss-import": "^12.0.0",
"postcss-loader": "^2.1.6",
"mini-css-extract-plugin": "^0.8.0",
"nightwatch": "^1.2.1",
"node-sass": "^4.12.0",
"postcss-css-variables": "^0.13.0",
"postcss-import": "^12.0.1",
"postcss-loader": "^3.0.0",
"prompt": "^1.0.0",
"raw-loader": "^0.5.1",
"sass-loader": "^7.1.0",
"sitemap": "^1.13.0",
"style-loader": "^0.21.0",
"url-loader": "^1.0.1",
"web-resource-inliner": "^4.2.1",
"webpack": "^4.16.4",
"webpack-dev-server": "^3.1.5",
"sass-loader": "^8.0.0",
"sitemap": "^4.1.1",
"style-loader": "^1.0.0",
"svg-url-loader": "^3.0.1",
"url-loader": "^2.1.0",
"webpack": "^4.39.3",
"webpack-bundle-analyzer": "^3.4.1",
"webpack-dev-server": "^3.8.0",
"webpack-node-externals": "^1.7.2",
"worker-loader": "^2.0.0"
},
"dependencies": {
"@babel/polyfill": "^7.4.4",
"@babel/runtime": "^7.5.5",
"arrive": "^2.4.1",
"babel-plugin-transform-builtin-extend": "1.1.2",
"babel-polyfill": "^6.26.0",
"bcryptjs": "^2.4.3",
"bignumber.js": "^7.2.1",
"bootstrap-colorpicker": "^2.5.3",
"bootstrap-material-design": "^4.1.1",
"bson": "^3.0.2",
"bignumber.js": "^9.0.0",
"blakejs": "^1.1.0",
"bootstrap": "4.3.1",
"bootstrap-colorpicker": "^3.1.2",
"bootstrap-material-design": "^4.1.2",
"bson": "^4.0.2",
"chi-squared": "^1.1.0",
"crypto-api": "^0.8.0",
"clippyjs": "0.0.3",
"core-js": "^3.2.1",
"crypto-api": "^0.8.5",
"crypto-js": "^3.1.9-1",
"ctph.js": "0.0.5",
"diff": "^3.5.0",
"es6-promisify": "^6.0.0",
"escodegen": "^1.11.0",
"d3": "^5.11.0",
"d3-hexbin": "^0.2.2",
"diff": "^4.0.1",
"es6-promisify": "^6.0.2",
"escodegen": "^1.12.0",
"esm": "^3.2.25",
"esmangle": "^1.0.1",
"esprima": "^4.0.1",
"exif-parser": "^0.1.12",
"file-saver": "^1.3.8",
"highlight.js": "^9.12.0",
"jquery": "^3.3.1",
"file-saver": "^2.0.2",
"geodesy": "^1.1.3",
"highlight.js": "^9.15.10",
"jimp": "^0.6.4",
"jquery": "3.4.1",
"js-crc": "^0.2.0",
"js-sha3": "^0.7.0",
"jsbn": "^1.1.0",
"jsesc": "^2.5.1",
"jsonpath": "^1.0.0",
"js-sha3": "^0.8.0",
"jsesc": "^2.5.2",
"jsonpath": "^1.0.2",
"jsonwebtoken": "^8.5.1",
"jsqr": "^1.2.0",
"jsrsasign": "8.0.12",
"kbpgp": "^2.0.77",
"lodash": "^4.17.10",
"loglevel": "^1.6.1",
"kbpgp": "2.1.3",
"libbzip2-wasm": "0.0.4",
"libyara-wasm": "0.0.12",
"lodash": "^4.17.15",
"loglevel": "^1.6.3",
"loglevel-message-prefix": "^3.0.0",
"moment": "^2.22.2",
"moment-timezone": "^0.5.21",
"ngeohash": "^0.6.0",
"node-forge": "^0.7.5",
"markdown-it": "^9.1.0",
"moment": "^2.24.0",
"moment-timezone": "^0.5.26",
"ngeohash": "^0.6.3",
"node-forge": "^0.8.5",
"node-md6": "^0.1.0",
"notepack.io": "^2.1.3",
"nodom": "^2.2.0",
"notepack.io": "^2.2.0",
"nwmatcher": "^1.4.4",
"otp": "^0.1.3",
"popper.js": "^1.14.4",
"scryptsy": "^2.0.0",
"popper.js": "^1.15.0",
"qr-image": "^3.2.0",
"scryptsy": "^2.1.0",
"snackbarjs": "^1.1.0",
"sortablejs": "^1.7.0",
"split.js": "^1.3.5",
"sortablejs": "^1.9.0",
"split.js": "^1.5.11",
"ssdeep.js": "0.0.2",
"ua-parser-js": "^0.7.18",
"tesseract.js": "^2.0.0-alpha.15",
"ua-parser-js": "^0.7.20",
"utf8": "^3.0.0",
"vkbeautify": "^0.99.3",
"xmldom": "^0.1.27",
"xpath": "0.0.27",
"xregexp": "^4.2.0",
"xregexp": "^4.2.4",
"zlibjs": "^0.3.1"
},
"scripts": {
"start": "grunt dev",
"build": "grunt prod",
"repl": "node src/node/repl.js",
"test": "grunt test",
"docs": "grunt docs",
"test-node-consumer": "grunt testnodeconsumer",
"testui": "grunt testui",
"testuidev": "npx nightwatch --env=dev",
"lint": "grunt lint",
"newop": "node --experimental-modules src/core/config/scripts/newOperation.mjs",
"postinstall": "[ -f node_modules/crypto-api/src/crypto-api.mjs ] || npx j2m node_modules/crypto-api/src/crypto-api.js"
"newop": "node --experimental-modules src/core/config/scripts/newOperation.mjs"
}
}

8
postcss.config.js
View file

@ -1,13 +1,7 @@
module.exports = {
plugins: [
require("postcss-import"),
require("autoprefixer")({
browsers: [
"Chrome >= 40",
"Firefox >= 35",
"Edge >= 14"
]
}),
require("autoprefixer"),
require("postcss-css-variables")({
preserve: true
}),

84
src/core/Chef.mjs
View file

@ -4,9 +4,10 @@
* @license Apache-2.0
*/
import Dish from "./Dish";
import Recipe from "./Recipe";
import Dish from "./Dish.mjs";
import Recipe from "./Recipe.mjs";
import log from "loglevel";
import { isWorkerEnvironment } from "./Utils.mjs";
/**
* The main controller for CyberChef.
@ -28,8 +29,6 @@ class Chef {
* @param {Object[]} recipeConfig - The recipe configuration object
* @param {Object} options - The options object storing various user choices
* @param {boolean} options.attempHighlight - Whether or not to attempt highlighting
* @param {number} progress - The position in the recipe to start from
* @param {number} [step] - Whether to only execute one operation in the recipe
*
* @returns {Object} response
* @returns {string} response.result - The output of the recipe
@ -38,46 +37,20 @@ class Chef {
* @returns {number} response.duration - The number of ms it took to execute the recipe
* @returns {number} response.error - The error object thrown by a failed operation (false if no error)
*/
async bake(input, recipeConfig, options, progress, step) {
async bake(input, recipeConfig, options) {
log.debug("Chef baking");
const startTime = new Date().getTime(),
recipe = new Recipe(recipeConfig),
containsFc = recipe.containsFlowControl(),
notUTF8 = options && options.hasOwnProperty("treatAsUtf8") && !options.treatAsUtf8;
let error = false;
if (containsFc && ENVIRONMENT_IS_WORKER()) self.setOption("attemptHighlight", false);
// Clean up progress
if (progress >= recipeConfig.length) {
notUTF8 = options && "treatAsUtf8" in options && !options.treatAsUtf8;
let error = false,
progress = 0;
}
if (step) {
// Unset breakpoint on this step
recipe.setBreakpoint(progress, false);
// Set breakpoint on next step
recipe.setBreakpoint(progress + 1, true);
}
if (containsFc && isWorkerEnvironment()) self.setOption("attemptHighlight", false);
// If the previously run operation presented a different value to its
// normal output, we need to recalculate it.
if (recipe.lastOpPresented(progress)) {
progress = 0;
}
// If stepping with flow control, we have to start from the beginning
// but still want to skip all previous breakpoints
if (progress > 0 && containsFc) {
recipe.removeBreaksUpTo(progress);
progress = 0;
}
// If starting from scratch, load data
if (progress === 0) {
const type = input instanceof ArrayBuffer ? Dish.ARRAY_BUFFER : Dish.STRING;
this.dish.set(input, type);
}
// Load data
const type = input instanceof ArrayBuffer ? Dish.ARRAY_BUFFER : Dish.STRING;
this.dish.set(input, type);
try {
progress = await recipe.execute(this.dish, progress);
@ -89,23 +62,26 @@ class Chef {
progress = err.progress;
}
// Depending on the size of the output, we may send it back as a string or an ArrayBuffer.
// This can prevent unnecessary casting as an ArrayBuffer can be easily downloaded as a file.
// The threshold is specified in KiB.
const threshold = (options.ioDisplayThreshold || 1024) * 1024;
const returnType = this.dish.size > threshold ? Dish.ARRAY_BUFFER : Dish.STRING;
// Create a raw version of the dish, unpresented
const rawDish = this.dish.clone();
// Present the raw result
await recipe.present(this.dish);
// Depending on the size of the output, we may send it back as a string or an ArrayBuffer.
// This can prevent unnecessary casting as an ArrayBuffer can be easily downloaded as a file.
// The threshold is specified in KiB.
const threshold = (options.ioDisplayThreshold || 1024) * 1024;
const returnType =
this.dish.size > threshold ?
Dish.ARRAY_BUFFER :
this.dish.type === Dish.HTML ?
Dish.HTML :
Dish.STRING;
return {
dish: rawDish,
result: this.dish.type === Dish.HTML ?
await this.dish.get(Dish.HTML, notUTF8) :
await this.dish.get(returnType, notUTF8),
result: await this.dish.get(returnType, notUTF8),
type: Dish.enumLookup(this.dish.type),
progress: progress,
duration: new Date().getTime() - startTime,
@ -157,9 +133,9 @@ class Chef {
* @param {number} pos.end - The end offset.
* @returns {Object}
*/
calculateHighlights(recipeConfig, direction, pos) {
async calculateHighlights(recipeConfig, direction, pos) {
const recipe = new Recipe(recipeConfig);
const highlights = recipe.generateHighlightList();
const highlights = await recipe.generateHighlightList();
if (!highlights) return false;
@ -193,6 +169,18 @@ class Chef {
return await newDish.get(type);
}
/**
* Gets the title of a dish and returns it
*
* @param {Dish} dish
* @param {number} [maxLength=100]
* @returns {string}
*/
async getDishTitle(dish, maxLength=100) {
const newDish = new Dish(dish);
return await newDish.getTitle(maxLength);
}
}
export default Chef;

94
src/core/ChefWorker.js
View file

@ -6,10 +6,9 @@
* @license Apache-2.0
*/
import "babel-polyfill";
import Chef from "./Chef";
import Chef from "./Chef.mjs";
import OperationConfig from "./config/OperationConfig.json";
import OpModules from "./config/modules/OpModules";
import OpModules from "./config/modules/OpModules.mjs";
// Add ">" to the start of all log messages in the Chef Worker
import loglevelMessagePrefix from "loglevel-message-prefix";
@ -26,6 +25,8 @@ self.chef = new Chef();
self.OpModules = OpModules;
self.OperationConfig = OperationConfig;
self.inputNum = -1;
// Tell the app that the worker has loaded and is ready to operate
self.postMessage({
@ -36,6 +37,9 @@ self.postMessage({
/**
* Respond to message from parent thread.
*
* inputNum is optional and only used for baking multiple inputs.
* Defaults to -1 when one isn't sent with the bake message.
*
* Messages should have the following format:
* {
* action: "bake" | "silentBake",
@ -44,8 +48,9 @@ self.postMessage({
* recipeConfig: {[Object]},
* options: {Object},
* progress: {number},
* step: {boolean}
* } | undefined
* step: {boolean},
* [inputNum=-1]: {number}
* }
* }
*/
self.addEventListener("message", function(e) {
@ -63,6 +68,9 @@ self.addEventListener("message", function(e) {
case "getDishAs":
getDishAs(r.data);
break;
case "getDishTitle":
getDishTitle(r.data);
break;
case "docURL":
// Used to set the URL of the current document so that scripts can be
// imported into an inline worker.
@ -92,30 +100,35 @@ self.addEventListener("message", function(e) {
async function bake(data) {
// Ensure the relevant modules are loaded
self.loadRequiredModules(data.recipeConfig);
try {
self.inputNum = (data.inputNum !== undefined) ? data.inputNum : -1;
const response = await self.chef.bake(
data.input, // The user's input
data.recipeConfig, // The configuration of the recipe
data.options, // Options set by the user
data.progress, // The current position in the recipe
data.step // Whether or not to take one step or execute the whole recipe
data.options // Options set by the user
);
const transferable = (data.input instanceof ArrayBuffer) ? [data.input] : undefined;
self.postMessage({
action: "bakeComplete",
data: Object.assign(response, {
id: data.id
id: data.id,
inputNum: data.inputNum,
bakeId: data.bakeId
})
});
}, transferable);
} catch (err) {
self.postMessage({
action: "bakeError",
data: Object.assign(err, {
id: data.id
})
data: {
error: err.message || err,
id: data.id,
inputNum: data.inputNum
}
});
}
self.inputNum = -1;
}
@ -137,13 +150,33 @@ function silentBake(data) {
*/
async function getDishAs(data) {
const value = await self.chef.getDishAs(data.dish, data.type);
const transferable = (data.type === "ArrayBuffer") ? [value] : undefined;
self.postMessage({
action: "dishReturned",
data: {
value: value,
id: data.id
}
}, transferable);
}
/**
* Gets the dish title
*
* @param {object} data
* @param {Dish} data.dish
* @param {number} data.maxLength
* @param {number} data.id
*/
async function getDishTitle(data) {
const title = await self.chef.getDishTitle(data.dish, data.maxLength);
self.postMessage({
action: "dishReturned",
data: {
value: title,
id: data.id
}
});
}
@ -157,8 +190,8 @@ async function getDishAs(data) {
* @param {number} pos.start - The start offset.
* @param {number} pos.end - The end offset.
*/
function calculateHighlights(recipeConfig, direction, pos) {
pos = self.chef.calculateHighlights(recipeConfig, direction, pos);
async function calculateHighlights(recipeConfig, direction, pos) {
pos = await self.chef.calculateHighlights(recipeConfig, direction, pos);
self.postMessage({
action: "highlightsCalculated",
@ -176,10 +209,10 @@ self.loadRequiredModules = function(recipeConfig) {
recipeConfig.forEach(op => {
const module = self.OperationConfig[op.op].module;
if (!OpModules.hasOwnProperty(module)) {
if (!(module in OpModules)) {
log.info(`Loading ${module} module`);
self.sendStatusMessage(`Loading ${module} module`);
self.importScripts(`${self.docURL}/${module}.js`);
self.importScripts(`${self.docURL}/modules/${module}.js`);
self.sendStatusMessage("");
}
});
@ -194,7 +227,28 @@ self.loadRequiredModules = function(recipeConfig) {
self.sendStatusMessage = function(msg) {
self.postMessage({
action: "statusMessage",
data: msg
data: {
message: msg,
inputNum: self.inputNum
}
});
};
/**
* Send progress update to the app.
*
* @param {number} progress
* @param {number} total
*/
self.sendProgressMessage = function(progress, total) {
self.postMessage({
action: "progressMessage",
data: {
progress: progress,
total: total,
inputNum: self.inputNum
}
});
};

360
src/core/Dish.mjs
View file

@ -5,10 +5,22 @@
* @license Apache-2.0
*/
import Utils from "./Utils";
import Utils, { isNodeEnvironment } from "./Utils.mjs";
import DishError from "./errors/DishError.mjs";
import BigNumber from "bignumber.js";
import { detectFileType } from "./lib/FileType.mjs";
import log from "loglevel";
import DishByteArray from "./dishTypes/DishByteArray.mjs";
import DishBigNumber from "./dishTypes/DishBigNumber.mjs";
import DishFile from "./dishTypes/DishFile.mjs";
import DishHTML from "./dishTypes/DishHTML.mjs";
import DishJSON from "./dishTypes/DishJSON.mjs";
import DishListFile from "./dishTypes/DishListFile.mjs";
import DishNumber from "./dishTypes/DishNumber.mjs";
import DishString from "./dishTypes/DishString.mjs";
/**
* The data being operated on by each operation.
*/
@ -17,16 +29,27 @@ class Dish {
/**
* Dish constructor
*
* @param {Dish} [dish=null] - A dish to clone
* @param {Dish || *} [dishOrInput=null] - A dish to clone OR an object
* literal to make into a dish
* @param {Enum} [type=null] (optional) - A type to accompany object
* literal input
*/
constructor(dish=null) {
this.value = [];
this.type = Dish.BYTE_ARRAY;
constructor(dishOrInput=null, type = null) {
this.value = new ArrayBuffer(0);
this.type = Dish.ARRAY_BUFFER;
if (dish &&
dish.hasOwnProperty("value") &&
dish.hasOwnProperty("type")) {
this.set(dish.value, dish.type);
// Case: dishOrInput is dish object
if (dishOrInput &&
Object.prototype.hasOwnProperty.call(dishOrInput, "value") &&
Object.prototype.hasOwnProperty.call(dishOrInput, "type")) {
this.set(dishOrInput.value, dishOrInput.type);
// input and type defined separately
} else if (dishOrInput && type !== null) {
this.set(dishOrInput, type);
// No type declared, so infer it.
} else if (dishOrInput) {
const inferredType = Dish.typeEnum(dishOrInput.constructor.name);
this.set(dishOrInput, inferredType);
}
}
@ -55,13 +78,14 @@ class Dish {
case "big number":
return Dish.BIG_NUMBER;
case "json":
case "object": // object constructor name. To allow JSON input in node.
return Dish.JSON;
case "file":
return Dish.FILE;
case "list<file>":
return Dish.LIST_FILE;
default:
throw "Invalid data type string. No matching enum.";
throw new DishError("Invalid data type string. No matching enum.");
}
}
@ -93,11 +117,48 @@ class Dish {
case Dish.LIST_FILE:
return "List<File>";
default:
throw "Invalid data type enum. No matching type.";
throw new DishError("Invalid data type enum. No matching type.");
}
}
/**
* Returns the value of the data in the type format specified.
*
* If running in a browser, get is asynchronous.
*
* @param {number} type - The data type of value, see Dish enums.
* @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
* @returns {* | Promise} - (Broswer) A promise | (Node) value of dish in given type
*/
get(type, notUTF8=false) {
if (typeof type === "string") {
type = Dish.typeEnum(type);
}
if (this.type !== type) {
// Node environment => _translate is sync
if (isNodeEnvironment()) {
this._translate(type, notUTF8);
return this.value;
// Browser environment => _translate is async
} else {
return new Promise((resolve, reject) => {
this._translate(type, notUTF8)
.then(() => {
resolve(this.value);
})
.catch(reject);
});
}
}
return this.value;
}
/**
* Sets the data value and type and then validates them.
*
@ -116,116 +177,86 @@ class Dish {
this.type = type;
if (!this.valid()) {
const sample = Utils.truncate(JSON.stringify(this.value), 13);
throw "Data is not a valid " + Dish.enumLookup(type) + ": " + sample;
const sample = Utils.truncate(JSON.stringify(this.value), 25);
throw new DishError(`Data is not a valid ${Dish.enumLookup(type)}: ${sample}`);
}
}
/**
* Returns the value of the data in the type format specified.
* Returns the Dish as the given type, without mutating the original dish.
*
* If running in a browser, get is asynchronous.
*
* @Node
*
* @param {number} type - The data type of value, see Dish enums.
* @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
* @returns {*} - The value of the output data.
* @returns {Dish | Promise} - (Broswer) A promise | (Node) value of dish in given type
*/
async get(type, notUTF8=false) {
if (typeof type === "string") {
type = Dish.typeEnum(type);
}
if (this.type !== type) {
await this._translate(type, notUTF8);
}
return this.value;
presentAs(type, notUTF8=false) {
const clone = this.clone();
return clone.get(type, notUTF8);
}
/**
* Translates the data to the given type format.
*
* @param {number} toType - The data type of value, see Dish enums.
* @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
* Detects the MIME type of the current dish
* @returns {string}
*/
async _translate(toType, notUTF8=false) {
log.debug(`Translating Dish from ${Dish.enumLookup(this.type)} to ${Dish.enumLookup(toType)}`);
const byteArrayToStr = notUTF8 ? Utils.byteArrayToChars : Utils.byteArrayToUtf8;
detectDishType() {
const data = new Uint8Array(this.value.slice(0, 2048)),
types = detectFileType(data);
// Convert data to intermediate byteArray type
switch (this.type) {
case Dish.STRING:
this.value = this.value ? Utils.strToByteArray(this.value) : [];
break;
case Dish.NUMBER:
this.value = typeof this.value === "number" ? Utils.strToByteArray(this.value.toString()) : [];
break;
case Dish.HTML:
this.value = this.value ? Utils.strToByteArray(Utils.unescapeHtml(Utils.stripHtmlTags(this.value, true))) : [];
break;
case Dish.ARRAY_BUFFER:
// Array.from() would be nicer here, but it's slightly slower
this.value = Array.prototype.slice.call(new Uint8Array(this.value));
break;
case Dish.BIG_NUMBER:
this.value = this.value instanceof BigNumber ? Utils.strToByteArray(this.value.toFixed()) : [];
break;
case Dish.JSON:
this.value = this.value ? Utils.strToByteArray(JSON.stringify(this.value, null, 4)) : [];
break;
case Dish.FILE:
this.value = await Utils.readFile(this.value);
this.value = Array.prototype.slice.call(this.value);
break;
case Dish.LIST_FILE:
this.value = await Promise.all(this.value.map(async f => Utils.readFile(f)));
this.value = this.value.map(b => Array.prototype.slice.call(b));
this.value = [].concat.apply([], this.value);
break;
default:
break;
}
this.type = Dish.BYTE_ARRAY;
// Convert from byteArray to toType
switch (toType) {
case Dish.STRING:
case Dish.HTML:
this.value = this.value ? byteArrayToStr(this.value) : "";
this.type = Dish.STRING;
break;
case Dish.NUMBER:
this.value = this.value ? parseFloat(byteArrayToStr(this.value)) : 0;
this.type = Dish.NUMBER;
break;
case Dish.ARRAY_BUFFER:
this.value = new Uint8Array(this.value).buffer;
this.type = Dish.ARRAY_BUFFER;
break;
case Dish.BIG_NUMBER:
try {
this.value = new BigNumber(byteArrayToStr(this.value));
} catch (err) {
this.value = new BigNumber(NaN);
}
this.type = Dish.BIG_NUMBER;
break;
case Dish.JSON:
this.value = JSON.parse(byteArrayToStr(this.value));
this.type = Dish.JSON;
break;
case Dish.FILE:
this.value = new File(this.value, "unknown");
break;
case Dish.LIST_FILE:
this.value = [new File(this.value, "unknown")];
this.type = Dish.LIST_FILE;
break;
default:
break;
if (!types.length || !types[0].mime || !types[0].mime === "text/plain") {
return null;
} else {
return types[0].mime;
}
}
/**
* Returns the title of the data up to the specified length
*
* @param {number} maxLength - The maximum title length
* @returns {string}
*/
async getTitle(maxLength) {
let title = "";
let cloned;
switch (this.type) {
case Dish.FILE:
title = this.value.name;
break;
case Dish.LIST_FILE:
title = `${this.value.length} file(s)`;
break;
case Dish.JSON:
title = "application/json";
break;
case Dish.NUMBER:
case Dish.BIG_NUMBER:
title = this.value.toString();
break;
case Dish.ARRAY_BUFFER:
case Dish.BYTE_ARRAY:
title = this.detectDishType();
if (title !== null) break;
// fall through if no mime type was detected
default:
try {
cloned = this.clone();
cloned.value = cloned.value.slice(0, 256);
title = await cloned.get(Dish.STRING);
} catch (err) {
log.error(`${Dish.enumLookup(this.type)} cannot be sliced. ${err}`);
}
}
return title.slice(0, maxLength);
}
/**
* Validates that the value is the type that has been specified.
* May have to disable parts of BYTE_ARRAY validation if it effects performance.
@ -235,7 +266,7 @@ class Dish {
valid() {
switch (this.type) {
case Dish.BYTE_ARRAY:
if (!(this.value instanceof Array)) {
if (!(this.value instanceof Uint8Array) && !(this.value instanceof Array)) {
return false;
}
@ -256,7 +287,21 @@ class Dish {
case Dish.ARRAY_BUFFER:
return this.value instanceof ArrayBuffer;
case Dish.BIG_NUMBER:
return this.value instanceof BigNumber;
if (BigNumber.isBigNumber(this.value)) return true;
/*
If a BigNumber is passed between WebWorkers it is serialised as a JSON
object with a coefficient (c), exponent (e) and sign (s). We detect this
and reinitialise it as a BigNumber object.
*/
if (Object.keys(this.value).sort().equals(["c", "e", "s"])) {
const temp = new BigNumber();
temp.c = this.value.c;
temp.e = this.value.e;
temp.s = this.value.s;
this.value = temp;
return true;
}
return false;
case Dish.JSON:
// All values can be serialised in some manner, so we return true in all cases
return true;
@ -357,12 +402,115 @@ class Dish {
);
break;
default:
throw new Error("Cannot clone Dish, unknown type");
throw new DishError("Cannot clone Dish, unknown type");
}
return newDish;
}
/**
* Translates the data to the given type format.
*
* If running in the browser, _translate is asynchronous.
*
* @param {number} toType - The data type of value, see Dish enums.
* @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
* @returns {Promise || undefined}
*/
_translate(toType, notUTF8=false) {
log.debug(`Translating Dish from ${Dish.enumLookup(this.type)} to ${Dish.enumLookup(toType)}`);
// Node environment => translate is sync
if (isNodeEnvironment()) {
this._toArrayBuffer();
this.type = Dish.ARRAY_BUFFER;
this._fromArrayBuffer(toType, notUTF8);
// Browser environment => translate is async
} else {
return new Promise((resolve, reject) => {
this._toArrayBuffer()
.then(() => this.type = Dish.ARRAY_BUFFER)
.then(() => {
this._fromArrayBuffer(toType);
resolve();
})
.catch(reject);
});
}
}
/**
* Convert this.value to an ArrayBuffer
*
* If running in a browser, _toByteArray is asynchronous.
*
* @returns {Promise || undefined}
*/
_toArrayBuffer() {
// Using 'bind' here to allow this.value to be mutated within translation functions
const toByteArrayFuncs = {
browser: {
[Dish.STRING]: () => Promise.resolve(DishString.toArrayBuffer.bind(this)()),
[Dish.NUMBER]: () => Promise.resolve(DishNumber.toArrayBuffer.bind(this)()),
[Dish.HTML]: () => Promise.resolve(DishHTML.toArrayBuffer.bind(this)()),
[Dish.ARRAY_BUFFER]: () => Promise.resolve(),
[Dish.BIG_NUMBER]: () => Promise.resolve(DishBigNumber.toArrayBuffer.bind(this)()),
[Dish.JSON]: () => Promise.resolve(DishJSON.toArrayBuffer.bind(this)()),
[Dish.FILE]: () => DishFile.toArrayBuffer.bind(this)(),
[Dish.LIST_FILE]: () => Promise.resolve(DishListFile.toArrayBuffer.bind(this)()),
[Dish.BYTE_ARRAY]: () => Promise.resolve(DishByteArray.toArrayBuffer.bind(this)()),
},
node: {
[Dish.STRING]: () => DishString.toArrayBuffer.bind(this)(),
[Dish.NUMBER]: () => DishNumber.toArrayBuffer.bind(this)(),
[Dish.HTML]: () => DishHTML.toArrayBuffer.bind(this)(),
[Dish.ARRAY_BUFFER]: () => {},
[Dish.BIG_NUMBER]: () => DishBigNumber.toArrayBuffer.bind(this)(),
[Dish.JSON]: () => DishJSON.toArrayBuffer.bind(this)(),
[Dish.FILE]: () => DishFile.toArrayBuffer.bind(this)(),
[Dish.LIST_FILE]: () => DishListFile.toArrayBuffer.bind(this)(),
[Dish.BYTE_ARRAY]: () => DishByteArray.toArrayBuffer.bind(this)(),
}
};
try {
return toByteArrayFuncs[isNodeEnvironment() && "node" || "browser"][this.type]();
} catch (err) {
throw new DishError(`Error translating from ${Dish.enumLookup(this.type)} to ArrayBuffer: ${err}`);
}
}
/**
* Convert this.value to the given type from ArrayBuffer
*
* @param {number} toType - the Dish enum to convert to
* @param {boolean} [notUTF8=false] - Do not treat strings as UTF8.
*/
_fromArrayBuffer(toType, notUTF8) {
// Using 'bind' here to allow this.value to be mutated within translation functions
const toTypeFunctions = {
[Dish.STRING]: () => DishString.fromArrayBuffer.bind(this)(notUTF8),
[Dish.NUMBER]: () => DishNumber.fromArrayBuffer.bind(this)(notUTF8),
[Dish.HTML]: () => DishHTML.fromArrayBuffer.bind(this)(notUTF8),
[Dish.ARRAY_BUFFER]: () => {},
[Dish.BIG_NUMBER]: () => DishBigNumber.fromArrayBuffer.bind(this)(notUTF8),
[Dish.JSON]: () => DishJSON.fromArrayBuffer.bind(this)(notUTF8),
[Dish.FILE]: () => DishFile.fromArrayBuffer.bind(this)(),
[Dish.LIST_FILE]: () => DishListFile.fromArrayBuffer.bind(this)(),
[Dish.BYTE_ARRAY]: () => DishByteArray.fromArrayBuffer.bind(this)(),
};
try {
toTypeFunctions[toType]();
this.type = toType;
} catch (err) {
throw new DishError(`Error translating from ArrayBuffer to ${Dish.enumLookup(toType)}: ${err}`);
}
}
}

15
src/core/Ingredient.mjs
View file

@ -4,8 +4,8 @@
* @license Apache-2.0
*/
import Utils from "./Utils";
import {fromHex} from "./lib/Hex";
import Utils from "./Utils.mjs";
import {fromHex} from "./lib/Hex.mjs";
/**
* The arguments to operations.
@ -23,8 +23,13 @@ class Ingredient {
this._value = null;
this.disabled = false;
this.hint = "";
this.rows = 0;
this.toggleValues = [];
this.target = null;
this.defaultIndex = 0;
this.min = null;
this.max = null;
this.step = 1;
if (ingredientConfig) {
this._parseConfig(ingredientConfig);
@ -44,8 +49,13 @@ class Ingredient {
this.defaultValue = ingredientConfig.value;
this.disabled = !!ingredientConfig.disabled;
this.hint = ingredientConfig.hint || false;
this.rows = ingredientConfig.rows || false;
this.toggleValues = ingredientConfig.toggleValues;
this.target = typeof ingredientConfig.target !== "undefined" ? ingredientConfig.target : null;
this.defaultIndex = typeof ingredientConfig.defaultIndex !== "undefined" ? ingredientConfig.defaultIndex : 0;
this.min = ingredientConfig.min;
this.max = ingredientConfig.max;
this.step = ingredientConfig.step;
}
@ -93,6 +103,7 @@ class Ingredient {
case "binaryString":
case "binaryShortString":
case "editableOption":
case "editableOptionShort":
return Utils.parseEscapedChars(data);
case "byteArray":
if (typeof data == "string") {

31
src/core/Operation.mjs
View file

@ -4,8 +4,8 @@
* @license Apache-2.0
*/
import Dish from "./Dish";
import Ingredient from "./Ingredient";
import Dish from "./Dish.mjs";
import Ingredient from "./Ingredient.mjs";
/**
* The Operation specified by the user to be run.
@ -23,6 +23,7 @@ class Operation {
this._breakpoint = false;
this._disabled = false;
this._flowControl = false;
this._manualBake = false;
this._ingList = [];
// Public fields
@ -179,8 +180,13 @@ class Operation {
if (ing.toggleValues) conf.toggleValues = ing.toggleValues;
if (ing.hint) conf.hint = ing.hint;
if (ing.rows) conf.rows = ing.rows;
if (ing.disabled) conf.disabled = ing.disabled;
if (ing.target) conf.target = ing.target;
if (ing.defaultIndex) conf.defaultIndex = ing.defaultIndex;
if (typeof ing.min === "number") conf.min = ing.min;
if (typeof ing.max === "number") conf.max = ing.max;
if (ing.step) conf.step = ing.step;
return conf;
});
}
@ -280,6 +286,7 @@ class Operation {
return this._flowControl;
}
/**
* Set whether this Operation is a flowcontrol op.
*
@ -289,6 +296,26 @@ class Operation {
this._flowControl = !!value;
}
/**
* Returns true if this Operation should not trigger AutoBake.
*
* @returns {boolean}
*/
get manualBake() {
return this._manualBake;
}
/**
* Set whether this Operation should trigger AutoBake.
*
* @param {boolean} value
*/
set manualBake(value) {
this._manualBake = !!value;
}
}
export default Operation;

92
src/core/Recipe.mjs
View file

@ -4,11 +4,15 @@
* @license Apache-2.0
*/
// import Operation from "./Operation.js";
import OpModules from "./config/modules/OpModules";
import OperationConfig from "./config/OperationConfig.json";
import OperationError from "./errors/OperationError";
import OperationError from "./errors/OperationError.mjs";
import Operation from "./Operation.mjs";
import DishError from "./errors/DishError.mjs";
import log from "loglevel";
import { isWorkerEnvironment } from "./Utils.mjs";
// Cache container for modules
let modules = null;
/**
* The Recipe controls a list of Operations and the Dish they operate on.
@ -36,16 +40,43 @@ class Recipe {
* @param {Object} recipeConfig
*/
_parseConfig(recipeConfig) {
for (let c = 0; c < recipeConfig.length; c++) {
const operationName = recipeConfig[c].op;
const opConf = OperationConfig[operationName];
const opObj = OpModules[opConf.module][operationName];
const operation = new opObj();
operation.ingValues = recipeConfig[c].args;
operation.breakpoint = recipeConfig[c].breakpoint;
operation.disabled = recipeConfig[c].disabled;
this.addOperation(operation);
recipeConfig.forEach(c => {
this.opList.push({
name: c.op,
module: OperationConfig[c.op].module,
ingValues: c.args,
breakpoint: c.breakpoint,
disabled: c.disabled,
});
});
}
/**
* Populate elements of opList with operation instances.
* Dynamic import here removes top-level cyclic dependency issue.
*
* @private
*/
async _hydrateOpList() {
if (!modules) {
// Using Webpack Magic Comments to force the dynamic import to be included in the main chunk
// https://webpack.js.org/api/module-methods/
modules = await import(/* webpackMode: "eager" */ "./config/modules/OpModules.mjs");
modules = modules.default;
}
this.opList = this.opList.map(o => {
if (o instanceof Operation) {
return o;
} else {
const op = new modules[o.module][o.name]();
op.ingValues = o.ingValues;
op.breakpoint = o.breakpoint;
op.disabled = o.disabled;
return op;
}
});
}
@ -55,7 +86,10 @@ class Recipe {
* @returns {Object[]}
*/
get config() {
return this.opList.map(op => op.config);
return this.opList.map(op => ({
op: op.name,
args: op.ingValues,
}));
}
@ -75,7 +109,19 @@ class Recipe {
* @param {Operation[]} operations
*/
addOperations(operations) {
this.opList = this.opList.concat(operations);
operations.forEach(o => {
if (o instanceof Operation) {
this.opList.push(o);
} else {
this.opList.push({
name: o.name,
module: o.module,
ingValues: o.args,
breakpoint: o.breakpoint,
disabled: o.disabled,
});
}
});
}
@ -108,7 +154,7 @@ class Recipe {
/**
* Returns true if there is an Flow Control Operation in this Recipe.
* Returns true if there is a Flow Control Operation in this Recipe.
*
* @returns {boolean}
*/
@ -137,6 +183,8 @@ class Recipe {
if (startFrom === 0) this.lastRunOp = null;
await this._hydrateOpList();
log.debug(`[*] Executing recipe of ${this.opList.length} operations, starting at ${startFrom}`);
for (let i = startFrom; i < this.opList.length; i++) {
@ -153,7 +201,12 @@ class Recipe {
try {
input = await dish.get(op.inputType);
log.debug("Executing operation");
log.debug(`Executing operation '${op.name}'`);
if (isWorkerEnvironment()) {
self.sendStatusMessage(`Baking... (${i+1}/${this.opList.length})`);
self.sendProgressMessage(i + 1, this.opList.length);
}
if (op.flowControl) {
// Package up the current state
@ -183,6 +236,10 @@ class Recipe {
// native types is not fully supported yet.
dish.set(err.message, "string");
return i;
} else if (err instanceof DishError ||
(err.type && err.type === "DishError")) {
dish.set(err.message, "string");
return i;
} else {
const e = typeof err == "string" ? { message: err } : err;
@ -250,7 +307,8 @@ class Recipe {
* @returns {function} highlights[].b
* @returns {Object[]} highlights[].args
*/
generateHighlightList() {
async generateHighlightList() {
await this._hydrateOpList();
const highlights = [];
for (let i = 0; i < this.opList.length; i++) {

314
src/core/Utils.mjs
View file

@ -5,11 +5,10 @@
*/
import utf8 from "utf8";
import moment from "moment-timezone";
import {fromBase64} from "./lib/Base64";
import {fromHex} from "./lib/Hex";
import {fromDecimal} from "./lib/Decimal";
import {fromBase64, toBase64} from "./lib/Base64.mjs";
import {fromHex} from "./lib/Hex.mjs";
import {fromDecimal} from "./lib/Decimal.mjs";
import {fromBinary} from "./lib/Binary.mjs";
/**
* Utility functions for use in operations, the core framework and the stage.
@ -95,7 +94,7 @@ class Utils {
const paddedBytes = new Array(numBytes);
paddedBytes.fill(padByte);
Array.prototype.map.call(arr, function(b, i) {
[...arr].forEach((b, i) => {
paddedBytes[i] = b;
});
@ -174,10 +173,11 @@ class Utils {
* @returns {string}
*/
static printable(str, preserveWs=false) {
if (ENVIRONMENT_IS_WEB() && window.app && !window.app.options.treatAsUtf8) {
if (isWebEnvironment() && window.app && !window.app.options.treatAsUtf8) {
str = Utils.byteArrayToChars(Utils.strToByteArray(str));
}
// eslint-disable-next-line no-misleading-character-class
const re = /[\0-\x08\x0B-\x0C\x0E-\x1F\x7F-\x9F\xAD\u0378\u0379\u037F-\u0383\u038B\u038D\u03A2\u0528-\u0530\u0557\u0558\u0560\u0588\u058B-\u058E\u0590\u05C8-\u05CF\u05EB-\u05EF\u05F5-\u0605\u061C\u061D\u06DD\u070E\u070F\u074B\u074C\u07B2-\u07BF\u07FB-\u07FF\u082E\u082F\u083F\u085C\u085D\u085F-\u089F\u08A1\u08AD-\u08E3\u08FF\u0978\u0980\u0984\u098D\u098E\u0991\u0992\u09A9\u09B1\u09B3-\u09B5\u09BA\u09BB\u09C5\u09C6\u09C9\u09CA\u09CF-\u09D6\u09D8-\u09DB\u09DE\u09E4\u09E5\u09FC-\u0A00\u0A04\u0A0B-\u0A0E\u0A11\u0A12\u0A29\u0A31\u0A34\u0A37\u0A3A\u0A3B\u0A3D\u0A43-\u0A46\u0A49\u0A4A\u0A4E-\u0A50\u0A52-\u0A58\u0A5D\u0A5F-\u0A65\u0A76-\u0A80\u0A84\u0A8E\u0A92\u0AA9\u0AB1\u0AB4\u0ABA\u0ABB\u0AC6\u0ACA\u0ACE\u0ACF\u0AD1-\u0ADF\u0AE4\u0AE5\u0AF2-\u0B00\u0B04\u0B0D\u0B0E\u0B11\u0B12\u0B29\u0B31\u0B34\u0B3A\u0B3B\u0B45\u0B46\u0B49\u0B4A\u0B4E-\u0B55\u0B58-\u0B5B\u0B5E\u0B64\u0B65\u0B78-\u0B81\u0B84\u0B8B-\u0B8D\u0B91\u0B96-\u0B98\u0B9B\u0B9D\u0BA0-\u0BA2\u0BA5-\u0BA7\u0BAB-\u0BAD\u0BBA-\u0BBD\u0BC3-\u0BC5\u0BC9\u0BCE\u0BCF\u0BD1-\u0BD6\u0BD8-\u0BE5\u0BFB-\u0C00\u0C04\u0C0D\u0C11\u0C29\u0C34\u0C3A-\u0C3C\u0C45\u0C49\u0C4E-\u0C54\u0C57\u0C5A-\u0C5F\u0C64\u0C65\u0C70-\u0C77\u0C80\u0C81\u0C84\u0C8D\u0C91\u0CA9\u0CB4\u0CBA\u0CBB\u0CC5\u0CC9\u0CCE-\u0CD4\u0CD7-\u0CDD\u0CDF\u0CE4\u0CE5\u0CF0\u0CF3-\u0D01\u0D04\u0D0D\u0D11\u0D3B\u0D3C\u0D45\u0D49\u0D4F-\u0D56\u0D58-\u0D5F\u0D64\u0D65\u0D76-\u0D78\u0D80\u0D81\u0D84\u0D97-\u0D99\u0DB2\u0DBC\u0DBE\u0DBF\u0DC7-\u0DC9\u0DCB-\u0DCE\u0DD5\u0DD7\u0DE0-\u0DF1\u0DF5-\u0E00\u0E3B-\u0E3E\u0E5C-\u0E80\u0E83\u0E85\u0E86\u0E89\u0E8B\u0E8C\u0E8E-\u0E93\u0E98\u0EA0\u0EA4\u0EA6\u0EA8\u0EA9\u0EAC\u0EBA\u0EBE\u0EBF\u0EC5\u0EC7\u0ECE\u0ECF\u0EDA\u0EDB\u0EE0-\u0EFF\u0F48\u0F6D-\u0F70\u0F98\u0FBD\u0FCD\u0FDB-\u0FFF\u10C6\u10C8-\u10CC\u10CE\u10CF\u1249\u124E\u124F\u1257\u1259\u125E\u125F\u1289\u128E\u128F\u12B1\u12B6\u12B7\u12BF\u12C1\u12C6\u12C7\u12D7\u1311\u1316\u1317\u135B\u135C\u137D-\u137F\u139A-\u139F\u13F5-\u13FF\u169D-\u169F\u16F1-\u16FF\u170D\u1715-\u171F\u1737-\u173F\u1754-\u175F\u176D\u1771\u1774-\u177F\u17DE\u17DF\u17EA-\u17EF\u17FA-\u17FF\u180F\u181A-\u181F\u1878-\u187F\u18AB-\u18AF\u18F6-\u18FF\u191D-\u191F\u192C-\u192F\u193C-\u193F\u1941-\u1943\u196E\u196F\u1975-\u197F\u19AC-\u19AF\u19CA-\u19CF\u19DB-\u19DD\u1A1C\u1A1D\u1A5F\u1A7D\u1A7E\u1A8A-\u1A8F\u1A9A-\u1A9F\u1AAE-\u1AFF\u1B4C-\u1B4F\u1B7D-\u1B7F\u1BF4-\u1BFB\u1C38-\u1C3A\u1C4A-\u1C4C\u1C80-\u1CBF\u1CC8-\u1CCF\u1CF7-\u1CFF\u1DE7-\u1DFB\u1F16\u1F17\u1F1E\u1F1F\u1F46\u1F47\u1F4E\u1F4F\u1F58\u1F5A\u1F5C\u1F5E\u1F7E\u1F7F\u1FB5\u1FC5\u1FD4\u1FD5\u1FDC\u1FF0\u1FF1\u1FF5\u1FFF\u200B-\u200F\u202A-\u202E\u2060-\u206F\u2072\u2073\u208F\u209D-\u209F\u20BB-\u20CF\u20F1-\u20FF\u218A-\u218F\u23F4-\u23FF\u2427-\u243F\u244B-\u245F\u2700\u2B4D-\u2B4F\u2B5A-\u2BFF\u2C2F\u2C5F\u2CF4-\u2CF8\u2D26\u2D28-\u2D2C\u2D2E\u2D2F\u2D68-\u2D6E\u2D71-\u2D7E\u2D97-\u2D9F\u2DA7\u2DAF\u2DB7\u2DBF\u2DC7\u2DCF\u2DD7\u2DDF\u2E3C-\u2E7F\u2E9A\u2EF4-\u2EFF\u2FD6-\u2FEF\u2FFC-\u2FFF\u3040\u3097\u3098\u3100-\u3104\u312E-\u3130\u318F\u31BB-\u31BF\u31E4-\u31EF\u321F\u32FF\u4DB6-\u4DBF\u9FCD-\u9FFF\uA48D-\uA48F\uA4C7-\uA4CF\uA62C-\uA63F\uA698-\uA69E\uA6F8-\uA6FF\uA78F\uA794-\uA79F\uA7AB-\uA7F7\uA82C-\uA82F\uA83A-\uA83F\uA878-\uA87F\uA8C5-\uA8CD\uA8DA-\uA8DF\uA8FC-\uA8FF\uA954-\uA95E\uA97D-\uA97F\uA9CE\uA9DA-\uA9DD\uA9E0-\uA9FF\uAA37-\uAA3F\uAA4E\uAA4F\uAA5A\uAA5B\uAA7C-\uAA7F\uAAC3-\uAADA\uAAF7-\uAB00\uAB07\uAB08\uAB0F\uAB10\uAB17-\uAB1F\uAB27\uAB2F-\uABBF\uABEE\uABEF\uABFA-\uABFF\uD7A4-\uD7AF\uD7C7-\uD7CA\uD7FC-\uD7FF\uE000-\uF8FF\uFA6E\uFA6F\uFADA-\uFAFF\uFB07-\uFB12\uFB18-\uFB1C\uFB37\uFB3D\uFB3F\uFB42\uFB45\uFBC2-\uFBD2\uFD40-\uFD4F\uFD90\uFD91\uFDC8-\uFDEF\uFDFE\uFDFF\uFE1A-\uFE1F\uFE27-\uFE2F\uFE53\uFE67\uFE6C-\uFE6F\uFE75\uFEFD-\uFF00\uFFBF-\uFFC1\uFFC8\uFFC9\uFFD0\uFFD1\uFFD8\uFFD9\uFFDD-\uFFDF\uFFE7\uFFEF-\uFFFB\uFFFE\uFFFF]/g;
const wsRe = /[\x09-\x10\x0D\u2028\u2029]/g;
@ -201,11 +201,20 @@ class Utils {
* Utils.parseEscapedChars("\\n");
*/
static parseEscapedChars(str) {
return str.replace(/(\\)?\\([bfnrtv0'"]|x[\da-fA-F]{2}|u[\da-fA-F]{4}|u\{[\da-fA-F]{1,6}\})/g, function(m, a, b) {
return str.replace(/(\\)?\\([bfnrtv'"]|[0-3][0-7]{2}|[0-7]{1,2}|x[\da-fA-F]{2}|u[\da-fA-F]{4}|u\{[\da-fA-F]{1,6}\}|\\)/g, function(m, a, b) {
if (a === "\\") return "\\"+b;
switch (b[0]) {
case "\\":
return "\\";
case "0":
return "\0";
case "1":
case "2":
case "3":
case "4":
case "5":
case "6":
case "7":
return String.fromCharCode(parseInt(b, 8));
case "b":
return "\b";
case "t":
@ -298,7 +307,7 @@ class Utils {
* Accepts hex, Base64, UTF8 and Latin1 strings.
*
* @param {string} str
* @param {string} type - One of "Hex", "Decimal", "Base64", "UTF8" or "Latin1"
* @param {string} type - One of "Binary", "Hex", "Decimal", "Base64", "UTF8" or "Latin1"
* @returns {byteArray}
*
* @example
@ -313,6 +322,8 @@ class Utils {
*/
static convertToByteArray(str, type) {
switch (type.toLowerCase()) {
case "binary":
return fromBinary(str);
case "hex":
return fromHex(str);
case "decimal":
@ -333,7 +344,7 @@ class Utils {
* Accepts hex, Base64, UTF8 and Latin1 strings.
*
* @param {string} str
* @param {string} type - One of "Hex", "Decimal", "Base64", "UTF8" or "Latin1"
* @param {string} type - One of "Binary", "Hex", "Decimal", "Base64", "UTF8" or "Latin1"
* @returns {string}
*
* @example
@ -348,6 +359,8 @@ class Utils {
*/
static convertToByteString(str, type) {
switch (type.toLowerCase()) {
case "binary":
return Utils.byteArrayToChars(fromBinary(str));
case "hex":
return Utils.byteArrayToChars(fromHex(str));
case "decimal":
@ -363,6 +376,61 @@ class Utils {
}
/**
* Converts a string to an ArrayBuffer.
* Treats the string as UTF-8 if any values are over 255.
*
* @param {string} str
* @returns {ArrayBuffer}
*
* @example
* // returns [72,101,108,108,111]
* Utils.strToArrayBuffer("Hello");
*
* // returns [228,189,160,229,165,189]
* Utils.strToArrayBuffer("你好");
*/
static strToArrayBuffer(str) {
const arr = new Uint8Array(str.length);
let i = str.length, b;
while (i--) {
b = str.charCodeAt(i);
arr[i] = b;
// If any of the bytes are over 255, read as UTF-8
if (b > 255) return Utils.strToUtf8ArrayBuffer(str);
}
return arr.buffer;
}
/**
* Converts a string to a UTF-8 ArrayBuffer.
*
* @param {string} str
* @returns {ArrayBuffer}
*
* @example
* // returns [72,101,108,108,111]
* Utils.strToUtf8ArrayBuffer("Hello");
*
* // returns [228,189,160,229,165,189]
* Utils.strToUtf8ArrayBuffer("你好");
*/
static strToUtf8ArrayBuffer(str) {
const utf8Str = utf8.encode(str);
if (str.length !== utf8Str.length) {
if (isWorkerEnvironment()) {
self.setOption("attemptHighlight", false);
} else if (isWebEnvironment()) {
window.app.options.attemptHighlight = false;
}
}
return Utils.strToArrayBuffer(utf8Str);
}
/**
* Converts a string to a byte array.
* Treats the string as UTF-8 if any values are over 255.
@ -407,9 +475,9 @@ class Utils {
const utf8Str = utf8.encode(str);
if (str.length !== utf8Str.length) {
if (ENVIRONMENT_IS_WORKER()) {
if (isWorkerEnvironment()) {
self.setOption("attemptHighlight", false);
} else if (ENVIRONMENT_IS_WEB()) {
} else if (isWebEnvironment()) {
window.app.options.attemptHighlight = false;
}
}
@ -455,7 +523,7 @@ class Utils {
/**
* Attempts to convert a byte array to a UTF-8 string.
*
* @param {byteArray} byteArray
* @param {byteArray|Uint8Array} byteArray
* @returns {string}
*
* @example
@ -469,11 +537,10 @@ class Utils {
const str = Utils.byteArrayToChars(byteArray);
try {
const utf8Str = utf8.decode(str);
if (str.length !== utf8Str.length) {
if (ENVIRONMENT_IS_WORKER()) {
if (isWorkerEnvironment()) {
self.setOption("attemptHighlight", false);
} else if (ENVIRONMENT_IS_WEB()) {
} else if (isWebEnvironment()) {
window.app.options.attemptHighlight = false;
}
}
@ -501,6 +568,7 @@ class Utils {
static byteArrayToChars(byteArray) {
if (!byteArray) return "";
let str = "";
// String concatenation appears to be faster than an array join
for (let i = 0; i < byteArray.length;) {
str += String.fromCharCode(byteArray[i++]);
}
@ -520,8 +588,8 @@ class Utils {
* Utils.arrayBufferToStr(Uint8Array.from([104,101,108,108,111]).buffer);
*/
static arrayBufferToStr(arrayBuffer, utf8=true) {
const byteArray = Array.prototype.slice.call(new Uint8Array(arrayBuffer));
return utf8 ? Utils.byteArrayToUtf8(byteArray) : Utils.byteArrayToChars(byteArray);
const arr = new Uint8Array(arrayBuffer);
return utf8 ? Utils.byteArrayToUtf8(arr) : Utils.byteArrayToChars(arr);
}
@ -555,8 +623,6 @@ class Utils {
if (renderNext) {
cell += b;
renderNext = false;
} else if (b === "\\") {
renderNext = true;
} else if (b === "\"" && !inString) {
inString = true;
} else if (b === "\"" && inString) {
@ -570,6 +636,10 @@ class Utils {
cell = "";
lines.push(line);
line = [];
// Skip next byte if it is also a line delim (e.g. \r\n)
if (lineDelims.indexOf(next) >= 0 && next !== b) {
i++;
}
} else {
cell += b;
}
@ -774,7 +844,7 @@ class Utils {
args = m[2]
.replace(/"/g, '\\"') // Escape double quotes
.replace(/(^|,|{|:)'/g, '$1"') // Replace opening ' with "
.replace(/([^\\]|[^\\]\\\\)'(,|:|}|$)/g, '$1"$2') // Replace closing ' with "
.replace(/([^\\]|(?:\\\\)+)'(,|:|}|$)/g, '$1"$2') // Replace closing ' with "
.replace(/\\'/g, "'"); // Unescape single quotes
args = "[" + args + "]";
@ -790,38 +860,6 @@ class Utils {
}
/**
* Expresses a number of milliseconds in a human readable format.
*
* Range | Sample Output
* -----------------------------|-------------------------------
* 0 to 45 seconds | a few seconds ago
* 45 to 90 seconds | a minute ago
* 90 seconds to 45 minutes | 2 minutes ago ... 45 minutes ago
* 45 to 90 minutes | an hour ago
* 90 minutes to 22 hours | 2 hours ago ... 22 hours ago
* 22 to 36 hours | a day ago
* 36 hours to 25 days | 2 days ago ... 25 days ago
* 25 to 45 days | a month ago
* 45 to 345 days | 2 months ago ... 11 months ago
* 345 to 545 days (1.5 years) | a year ago
* 546 days+ | 2 years ago ... 20 years ago
*
* @param {number} ms
* @returns {string}
*
* @example
* // returns "3 minutes"
* Utils.fuzzyTime(152435);
*
* // returns "5 days"
* Utils.fuzzyTime(456851321);
*/
static fuzzyTime(ms) {
return moment.duration(ms, "milliseconds").humanize();
}
/**
* Formats a list of files or directories.
*
@ -843,12 +881,24 @@ class Utils {
return html;
};
const formatContent = function (buff, type) {
if (type.startsWith("image")) {
let dataURI = "data:";
dataURI += type + ";";
dataURI += "base64," + toBase64(buff);
return "<img style='max-width: 100%;' src='" + dataURI + "'>";
} else {
return `<pre>${Utils.escapeHtml(Utils.arrayBufferToStr(buff.buffer))}</pre>`;
}
};
const formatFile = async function(file, i) {
const buff = await Utils.readFile(file);
const blob = new Blob(
[buff],
{type: "octet/stream"}
{type: file.type || "octet/stream"}
);
const blobURL = URL.createObjectURL(blob);
const html = `<div class='card' style='white-space: normal;'>
<div class='card-header' id='heading${i}'>
@ -863,16 +913,25 @@ class Utils {
<span class='float-right' style="margin-top: -3px">
${file.size.toLocaleString()} bytes
<a title="Download ${Utils.escapeHtml(file.name)}"
href='${URL.createObjectURL(blob)}'
download='${Utils.escapeHtml(file.name)}'>
href="${blobURL}"
download="${Utils.escapeHtml(file.name)}"
data-toggle="tooltip">
<i class="material-icons" style="vertical-align: bottom">save</i>
</a>
<a title="Move to input"
href="#"
blob-url="${blobURL}"
file-name="${Utils.escapeHtml(file.name)}"
class="extract-file"
data-toggle="tooltip">
<i class="material-icons" style="vertical-align: bottom">open_in_browser</i>
</a>
</span>
</h6>
</div>
<div id='collapse${i}' class='collapse' aria-labelledby='heading${i}' data-parent="#files">
<div class='card-body'>
<pre>${Utils.escapeHtml(Utils.arrayBufferToStr(buff.buffer))}</pre>
${formatContent(buff, file.type)}
</div>
</div>
</div>`;
@ -934,7 +993,7 @@ class Utils {
/**
* Reads a File and returns the data as a Uint8Array.
*
* @param {File} file
* @param {File | for node: array|arrayBuffer|buffer|string} file
* @returns {Uint8Array}
*
* @example
@ -942,33 +1001,57 @@ class Utils {
* await Utils.readFile(new File(["hello"], "test"))
*/
static readFile(file) {
return new Promise((resolve, reject) => {
const reader = new FileReader();
const data = new Uint8Array(file.size);
let offset = 0;
const CHUNK_SIZE = 10485760; // 10MiB
const seek = function() {
if (offset >= file.size) {
resolve(data);
return;
}
const slice = file.slice(offset, offset + CHUNK_SIZE);
reader.readAsArrayBuffer(slice);
};
if (isNodeEnvironment()) {
return Buffer.from(file).buffer;
} else {
return new Promise((resolve, reject) => {
const reader = new FileReader();
const data = new Uint8Array(file.size);
let offset = 0;
const CHUNK_SIZE = 10485760; // 10MiB
const seek = function() {
if (offset >= file.size) {
resolve(data);
return;
}
const slice = file.slice(offset, offset + CHUNK_SIZE);
reader.readAsArrayBuffer(slice);
};
reader.onload = function(e) {
data.set(new Uint8Array(reader.result), offset);
offset += CHUNK_SIZE;
seek();
};
reader.onerror = function(e) {
reject(reader.error.message);
};
reader.onload = function(e) {
data.set(new Uint8Array(reader.result), offset);
offset += CHUNK_SIZE;
seek();
};
});
}
}
reader.onerror = function(e) {
reject(reader.error.message);
};
/**
* Synchronously read the raw data from a File object.
*
* Only works in the Node environment
*
* @param {File} file - a File shim object (see src/node/File.mjs)
* @returns {ArrayBuffer} the data from the file in an ArrayBuffer
* @throws {TypeError} thrown if the method is called from a browser environment
*/
static readFileSync(file) {
if (!isNodeEnvironment()) {
throw new TypeError("Browser environment cannot support readFileSync");
}
seek();
});
const arrayBuffer = Uint8Array.from(file.data);
return arrayBuffer.buffer;
}
@ -1028,9 +1111,11 @@ class Utils {
static charRep(token) {
return {
"Space": " ",
"Percent": "%",
"Comma": ",",
"Semi-colon": ";",
"Colon": ":",
"Tab": "\t",
"Line feed": "\n",
"CRLF": "\r\n",
"Forward slash": "/",
@ -1052,6 +1137,7 @@ class Utils {
static regexRep(token) {
return {
"Space": /\s+/g,
"Percent": /%/g,
"Comma": /,/g,
"Semi-colon": /;/g,
"Colon": /:/g,
@ -1067,6 +1153,30 @@ class Utils {
}
/**
* Check whether the code is running in a Node.js environment
* @returns {boolean}
*/
export function isNodeEnvironment() {
return typeof process !== "undefined" && process.versions != null && process.versions.node != null;
}
/**
* Check whether the code is running in a web environment
* @returns {boolean}
*/
export function isWebEnvironment() {
return typeof window === "object";
}
/**
* Check whether the code is running in a worker
* @returns {boolean}
*/
export function isWorkerEnvironment() {
return typeof importScripts === "function";
}
export default Utils;
@ -1085,7 +1195,7 @@ export default Utils;
Array.prototype.unique = function() {
const u = {}, a = [];
for (let i = 0, l = this.length; i < l; i++) {
if (u.hasOwnProperty(this[i])) {
if (Object.prototype.hasOwnProperty.call(u, this[i])) {
continue;
}
a.push(this[i]);
@ -1178,6 +1288,46 @@ String.prototype.count = function(chr) {
};
/**
* Wrapper for self.sendStatusMessage to handle different environments.
*
* @param {string} msg
*/
export function sendStatusMessage(msg) {
if (isWorkerEnvironment())
self.sendStatusMessage(msg);
else if (isWebEnvironment())
app.alert(msg, 10000);
else if (isNodeEnvironment())
// eslint-disable-next-line no-console
console.debug(msg);
}
const debounceTimeouts = {};
/**
* Debouncer to stop functions from being executed multiple times in a
* short space of time
* https://davidwalsh.name/javascript-debounce-function
*
* @param {function} func - The function to be executed after the debounce time
* @param {number} wait - The time (ms) to wait before executing the function
* @param {string} id - Unique ID to reference the timeout for the function
* @param {object} scope - The object to bind to the debounced function
* @param {array} args - Array of arguments to be passed to func
* @returns {function}
*/
export function debounce(func, wait, id, scope, args) {
return function() {
const later = function() {
func.apply(scope, args);
};
clearTimeout(debounceTimeouts[id]);
debounceTimeouts[id] = setTimeout(later, wait);
};
}
/*
* Polyfills
*/

112
src/core/config/Categories.json
View file

@ -25,6 +25,8 @@
"From Base32",
"To Base58",
"From Base58",
"To Base62",
"From Base62",
"To Base85",
"From Base85",
"To Base",
@ -49,11 +51,15 @@
"Change IP format",
"Encode text",
"Decode text",
"Text Encoding Brute Force",
"Swap endianness",
"To MessagePack",
"From MessagePack",
"To Braille",
"From Braille"
"From Braille",
"Parse TLV",
"CSV to JSON",
"JSON to CSV"
]
},
{
@ -79,10 +85,14 @@
"Vigenère Decode",
"To Morse Code",
"From Morse Code",
"Bacon Cipher Encode",
"Bacon Cipher Decode",
"Bifid Cipher Encode",
"Bifid Cipher Decode",
"Affine Cipher Encode",
"Affine Cipher Decode",
"A1Z26 Cipher Encode",
"A1Z26 Cipher Decode",
"Atbash Cipher",
"Substitute",
"Derive PBKDF2 key",
@ -91,7 +101,16 @@
"Scrypt",
"Generate RSA Key Pair",
"RSA Sign",
"Pseudo-Random Number Generator"
"JWT Sign",
"JWT Verify",
"JWT Decode",
"Citrix CTX1 Encode",
"Citrix CTX1 Decode",
"Pseudo-Random Number Generator",
"Enigma",
"Bombe",
"Multiple Bombe",
"Typex"
]
},
{
@ -107,7 +126,8 @@
"PGP Encrypt",
"PGP Decrypt",
"PGP Encrypt and Sign",
"PGP Decrypt and Verify"
"PGP Decrypt and Verify",
"Parse SSH Host Key"
]
},
{
@ -144,20 +164,28 @@
"name": "Networking",
"ops": [
"HTTP request",
"DNS over HTTPS",
"Strip HTTP headers",
"Dechunk HTTP response",
"Parse User Agent",
"Parse IP range",
"Parse IPv6 address",
"Parse IPv4 header",
"Parse UDP",
"Parse SSH Host Key",
"Parse URI",
"URL Encode",
"URL Decode",
"Protobuf Decode",
"VarInt Encode",
"VarInt Decode",
"Format MAC addresses",
"Change IP format",
"Group IP addresses",
"Encode NetBIOS Name",
"Decode NetBIOS Name"
"Decode NetBIOS Name",
"Defang URL",
"Defang IP Addresses"
]
},
{
@ -165,6 +193,7 @@
"ops": [
"Encode text",
"Decode text",
"Remove Diacritics",
"Unescape Unicode Characters"
]
},
@ -176,6 +205,8 @@
"Remove null bytes",
"To Upper case",
"To Lower case",
"To Case Insensitive Regex",
"From Case Insensitive Regex",
"Add line numbers",
"Remove line numbers",
"To Table",
@ -200,6 +231,8 @@
"Convert mass",
"Convert speed",
"Convert data units",
"Convert co-ordinate format",
"Show on map",
"Parse UNIX file permissions",
"Swap endianness",
"Parse colour code",
@ -237,7 +270,8 @@
"XPath expression",
"JPath expression",
"CSS selector",
"Extract EXIF"
"Extract EXIF",
"Extract Files"
]
},
{
@ -275,6 +309,10 @@
"HAS-160",
"Whirlpool",
"Snefru",
"BLAKE2b",
"BLAKE2s",
"GOST hash",
"Streebog",
"SSDEEP",
"CTPH",
"Compare SSDEEP hashes",
@ -289,11 +327,10 @@
"Fletcher-32 Checksum",
"Fletcher-64 Checksum",
"Adler-32 Checksum",
"CRC-8 Checksum",
"CRC-16 Checksum",
"CRC-32 Checksum",
"TCP/IP Checksum",
"To Geohash",
"From Geohash"
"TCP/IP Checksum"
]
},
{
@ -325,7 +362,53 @@
"BSON serialise",
"BSON deserialise",
"To MessagePack",
"From MessagePack"
"From MessagePack",
"Render Markdown"
]
},
{
"name": "Forensics",
"ops": [
"Detect File Type",
"Scan for Embedded Files",
"Extract Files",
"Remove EXIF",
"Extract EXIF",
"Extract RGBA",
"View Bit Plane",
"Randomize Colour Palette",
"Extract LSB"
]
},
{
"name": "Multimedia",
"ops": [
"Render Image",
"Play Media",
"Optical Character Recognition",
"Remove EXIF",
"Extract EXIF",
"Split Colour Channels",
"Rotate Image",
"Resize Image",
"Blur Image",
"Dither Image",
"Invert Image",
"Flip Image",
"Crop Image",
"Image Brightness / Contrast",
"Image Opacity",
"Image Filter",
"Contain Image",
"Cover Image",
"Image Hue/Saturation/Lightness",
"Sharpen Image",
"Convert Image Format",
"Add Text To Image",
"Hex Density chart",
"Scatter chart",
"Series chart",
"Heatmap chart"
]
},
{
@ -333,18 +416,18 @@
"ops": [
"Entropy",
"Frequency distribution",
"Index of Coincidence",
"Chi Square",
"Detect File Type",
"Scan for Embedded Files",
"Disassemble x86",
"Pseudo-Random Number Generator",
"Generate UUID",
"Generate TOTP",
"Generate HOTP",
"Generate QR Code",
"Parse QR Code",
"Haversine distance",
"Render Image",
"Remove EXIF",
"Extract EXIF",
"HTML To Text",
"Generate Lorem Ipsum",
"Numberwang",
"XKCD Random Number"
]
@ -354,6 +437,7 @@
"ops": [
"Magic",
"Fork",
"Subsection",
"Merge",
"Register",
"Label",

11
src/core/config/scripts/generateConfig.mjs
View file

@ -14,7 +14,7 @@
import path from "path";
import fs from "fs";
import process from "process";
import * as Ops from "../../operations/index";
import * as Ops from "../../operations/index.mjs";
const dir = path.join(process.cwd() + "/src/core/config/");
if (!fs.existsSync(dir)) {
@ -41,14 +41,15 @@ for (const opObj in Ops) {
inputType: op.inputType,
outputType: op.presentType,
flowControl: op.flowControl,
manualBake: op.manualBake,
args: op.args
};
if (op.hasOwnProperty("patterns")) {
if ("patterns" in op) {
operationConfig[op.name].patterns = op.patterns;
}
if (!modules.hasOwnProperty(op.module))
if (!(op.module in modules))
modules[op.module] = {};
modules[op.module][op.name] = opObj;
}
@ -83,7 +84,7 @@ for (const module in modules) {
for (const opName in modules[module]) {
const objName = modules[module][opName];
code += `import ${objName} from "../../operations/${objName}";\n`;
code += `import ${objName} from "../../operations/${objName}.mjs";\n`;
}
code += `
@ -123,7 +124,7 @@ let opModulesCode = `/**
`;
for (const module in modules) {
opModulesCode += `import ${module}Module from "./${module}";\n`;
opModulesCode += `import ${module}Module from "./${module}.mjs";\n`;
}
opModulesCode += `

2
src/core/config/scripts/generateOpsIndex.mjs
View file

@ -39,7 +39,7 @@ let code = `/**
`;
opObjs.forEach(obj => {
code += `import ${obj} from "./${obj}";\n`;
code += `import ${obj} from "./${obj}.mjs";\n`;
});
code += `

8
src/core/config/scripts/newOperation.mjs
View file

@ -13,7 +13,7 @@ import colors from "colors";
import process from "process";
import fs from "fs";
import path from "path";
import EscapeString from "../../operations/EscapeString";
import EscapeString from "../../operations/EscapeString.mjs";
const dir = path.join(process.cwd() + "/src/core/operations/");
@ -130,8 +130,8 @@ prompt.get(schema, (err, result) => {
* @license Apache-2.0
*/
import Operation from "../Operation";
import OperationError from "../errors/OperationError";
import Operation from "../Operation.mjs";
import OperationError from "../errors/OperationError.mjs";
/**
* ${result.opName} operation
@ -222,7 +222,7 @@ export default ${moduleName};
console.log(`\nNext steps:
1. Add your operation to ${colors.green("src/core/config/Categories.json")}
2. Write your operation code.
3. Write tests in ${colors.green("test/tests/operations/")}
3. Write tests in ${colors.green("tests/operations/tests/")}
4. Run ${colors.cyan("npm run lint")} and ${colors.cyan("npm run test")}
5. Submit a Pull Request to get your operation added to the official CyberChef repository.`);

39
src/core/dishTypes/DishBigNumber.mjs Normal file
View file

@ -0,0 +1,39 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
import Utils from "../Utils.mjs";
import BigNumber from "bignumber.js";
/**
* translation methods for BigNumber Dishes
*/
class DishBigNumber extends DishType {
/**
* convert the given value to a ArrayBuffer
* @param {BigNumber} value
*/
static toArrayBuffer() {
DishBigNumber.checkForValue(this.value);
this.value = BigNumber.isBigNumber(this.value) ? Utils.strToArrayBuffer(this.value.toFixed()) : new ArrayBuffer;
}
/**
* convert the given value from a ArrayBuffer
* @param {boolean} notUTF8
*/
static fromArrayBuffer(notUTF8) {
DishBigNumber.checkForValue(this.value);
try {
this.value = new BigNumber(Utils.arrayBufferToStr(this.value, !notUTF8));
} catch (err) {
this.value = new BigNumber(NaN);
}
}
}
export default DishBigNumber;

31
src/core/dishTypes/DishByteArray.mjs Normal file
View file

@ -0,0 +1,31 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
/**
* Translation methods for ArrayBuffer Dishes
*/
class DishByteArray extends DishType {
/**
* convert the given value to a ArrayBuffer
*/
static toArrayBuffer() {
DishByteArray.checkForValue(this.value);
this.value = new Uint8Array(this.value).buffer;
}
/**
* convert the given value from a ArrayBuffer
*/
static fromArrayBuffer() {
DishByteArray.checkForValue(this.value);
this.value = Array.prototype.slice.call(new Uint8Array(this.value));
}
}
export default DishByteArray;

42
src/core/dishTypes/DishFile.mjs Normal file
View file

@ -0,0 +1,42 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
import Utils, { isNodeEnvironment } from "../Utils.mjs";
/**
* Translation methods for file Dishes
*/
class DishFile extends DishType {
/**
* convert the given value to an ArrayBuffer
* @param {File} value
*/
static toArrayBuffer() {
DishFile.checkForValue(this.value);
if (isNodeEnvironment()) {
this.value = Utils.readFileSync(this.value);
} else {
return new Promise((resolve, reject) => {
Utils.readFile(this.value)
.then(v => this.value = v.buffer)
.then(resolve)
.catch(reject);
});
}
}
/**
* convert the given value from an ArrayBuffer
*/
static fromArrayBuffer() {
DishFile.checkForValue(this.value);
this.value = new File(this.value, "unknown");
}
}
export default DishFile;

26
src/core/dishTypes/DishHTML.mjs Normal file
View file

@ -0,0 +1,26 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishString from "./DishString.mjs";
import Utils from "../Utils.mjs";
/**
* Translation methods for HTML Dishes
*/
class DishHTML extends DishString {
/**
* convert the given value to a ArrayBuffer
* @param {String} value
*/
static toArrayBuffer() {
DishHTML.checkForValue(this.value);
this.value = this.value ? Utils.strToArrayBuffer(Utils.unescapeHtml(Utils.stripHtmlTags(this.value, true))) : new ArrayBuffer;
}
}
export default DishHTML;

33
src/core/dishTypes/DishJSON.mjs Normal file
View file

@ -0,0 +1,33 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
import Utils from "../Utils.mjs";
/**
* Translation methods for JSON dishes
*/
class DishJSON extends DishType {
/**
* convert the given value to a ArrayBuffer
*/
static toArrayBuffer() {
DishJSON.checkForValue(this.value);
this.value = this.value ? Utils.strToArrayBuffer(JSON.stringify(this.value, null, 4)) : new ArrayBuffer;
}
/**
* convert the given value from a ArrayBuffer
* @param {boolean} notUTF8
*/
static fromArrayBuffer(notUTF8) {
DishJSON.checkForValue(this.value);
this.value = JSON.parse(Utils.arrayBufferToStr(this.value, !notUTF8));
}
}
export default DishJSON;

58
src/core/dishTypes/DishListFile.mjs Normal file
View file

@ -0,0 +1,58 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
import { isNodeEnvironment } from "../Utils.mjs";
/**
* Translation methods for ListFile Dishes
*/
class DishListFile extends DishType {
/**
* convert the given value to a ArrayBuffer
*/
static toArrayBuffer() {
DishListFile.checkForValue(this.value);
if (isNodeEnvironment()) {
this.value = this.value.map(file => Uint8Array.from(file.data));
}
this.value = DishListFile.concatenateTypedArrays(...this.value).buffer;
}
/**
* convert the given value from a ArrayBuffer
*/
static fromArrayBuffer() {
DishListFile.checkForValue(this.value);
this.value = [new File(this.value, "unknown")];
}
/**
* Concatenates a list of Uint8Arrays together
*
* @param {Uint8Array[]} arrays
* @returns {Uint8Array}
*/
static concatenateTypedArrays(...arrays) {
let totalLength = 0;
for (const arr of arrays) {
totalLength += arr.length;
}
const result = new Uint8Array(totalLength);
let offset = 0;
for (const arr of arrays) {
result.set(arr, offset);
offset += arr.length;
}
return result;
}
}
export default DishListFile;

34
src/core/dishTypes/DishNumber.mjs Normal file
View file

@ -0,0 +1,34 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
import Utils from "../Utils.mjs";
/**
* Translation methods for number dishes
*/
class DishNumber extends DishType {
/**
* convert the given value to a ArrayBuffer
*/
static toArrayBuffer() {
DishNumber.checkForValue(this.value);
this.value = typeof this.value === "number" ? Utils.strToArrayBuffer(this.value.toString()) : new ArrayBuffer;
}
/**
* convert the given value from a ArrayBuffer
* @param {boolean} notUTF8
*/
static fromArrayBuffer(notUTF8) {
DishNumber.checkForValue(this.value);
this.value = this.value ? parseFloat(Utils.arrayBufferToStr(this.value, !notUTF8)) : 0;
}
}
export default DishNumber;

34
src/core/dishTypes/DishString.mjs Normal file
View file

@ -0,0 +1,34 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishType from "./DishType.mjs";
import Utils from "../Utils.mjs";
/**
* Translation methods for string dishes
*/
class DishString extends DishType {
/**
* convert the given value to a ArrayBuffer
*/
static toArrayBuffer() {
DishString.checkForValue(this.value);
this.value = this.value ? Utils.strToArrayBuffer(this.value) : new ArrayBuffer;
}
/**
* convert the given value from a ArrayBuffer
* @param {boolean} notUTF8
*/
static fromArrayBuffer(notUTF8) {
DishString.checkForValue(this.value);
this.value = this.value ? Utils.arrayBufferToStr(this.value, !notUTF8) : "";
}
}
export default DishString;

39
src/core/dishTypes/DishType.mjs Normal file
View file

@ -0,0 +1,39 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
/**
* Abstract class for dish translation methods
*/
class DishType {
/**
* Warn translations dont work without value from bind
*/
static checkForValue(value) {
if (value === undefined) {
throw new Error("only use translation methods with .bind");
}
}
/**
* convert the given value to a ArrayBuffer
* @param {*} value
*/
static toArrayBuffer() {
throw new Error("toArrayBuffer has not been implemented");
}
/**
* convert the given value from a ArrayBuffer
* @param {boolean} notUTF8
*/
static fromArrayBuffer(notUTF8=undefined) {
throw new Error("fromArrayBuffer has not been implemented");
}
}
export default DishType;

26
src/core/dishTypes/index.mjs Normal file
View file

@ -0,0 +1,26 @@
/**
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import DishByteArray from "./DishByteArray.mjs";
import DishBigNumber from "./DishBigNumber.mjs";
import DishFile from "./DishFile.mjs";
import DishHTML from "./DishHTML.mjs";
import DishJSON from "./DishJSON.mjs";
import DishListFile from "./DishListFile.mjs";
import DishNumber from "./DishNumber.mjs";
import DishString from "./DishString.mjs";
export {
DishByteArray,
DishBigNumber,
DishFile,
DishHTML,
DishJSON,
DishListFile,
DishNumber,
DishString,
};

26
src/core/errors/DishError.mjs Normal file
View file

@ -0,0 +1,26 @@
/**
* Custom error type for handling Dish type errors.
* i.e. where the Dish cannot be successfully translated between types
*
* @author n1474335 [n1474335@gmail.com]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
class DishError extends Error {
/**
* Standard error constructor. Adds no new behaviour.
*
* @param args - Standard error args
*/
constructor(...args) {
super(...args);
this.type = "DishError";
if (Error.captureStackTrace) {
Error.captureStackTrace(this, DishError);
}
}
}
export default DishError;

25
src/core/errors/ExcludedOperationError.mjs Normal file
View file

@ -0,0 +1,25 @@
/**
* Custom error type for handling operation that isnt included in node.js API
*
* @author d98762625 [d98762625@gmail.com]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
class ExcludedOperationError extends Error {
/**
* Standard error constructor. Adds no new behaviour.
*
* @param args - Standard error args
*/
constructor(...args) {
super(...args);
this.type = "ExcludedOperationError";
if (Error.captureStackTrace) {
Error.captureStackTrace(this, ExcludedOperationError);
}
}
}
export default ExcludedOperationError;

4
src/core/lib/Arithmetic.mjs
View file

@ -5,7 +5,7 @@
* @license Apache-2.0
*/
import Utils from "../Utils";
import Utils from "../Utils.mjs";
import BigNumber from "bignumber.js";
@ -109,7 +109,7 @@ export function mean(data) {
*/
export function median(data) {
if ((data.length % 2) === 0 && data.length > 0) {
data.sort(function(a, b){
data.sort(function(a, b) {
return a.minus(b);
});
const first = data[Math.floor(data.length / 2)];

0
src/core/lib/BCD.mjs Executable file → Normal file
View file

66
src/core/lib/Bacon.mjs Normal file
View file

@ -0,0 +1,66 @@
/**
* Bacon Cipher resources.
*
* @author Karsten Silkenbäumer [github.com/kassi]
* @copyright Karsten Silkenbäumer 2019
* @license Apache-2.0
*/
/**
* Bacon definitions.
*/
export const BACON_ALPHABETS = {
"Standard (I=J and U=V)": {
alphabet: "ABCDEFGHIKLMNOPQRSTUWXYZ",
codes: [0, 1, 2, 3, 4, 5, 6, 7, 8, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 19, 20, 21, 22, 23]
},
"Complete": {
alphabet: "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
}
};
export const BACON_TRANSLATION_01 = "0/1";
export const BACON_TRANSLATION_AB = "A/B";
export const BACON_TRANSLATION_CASE = "Case";
export const BACON_TRANSLATION_AMNZ = "A-M/N-Z first letter";
export const BACON_TRANSLATIONS = [
BACON_TRANSLATION_01,
BACON_TRANSLATION_AB,
BACON_TRANSLATION_CASE,
BACON_TRANSLATION_AMNZ,
];
export const BACON_TRANSLATIONS_FOR_ENCODING = [
BACON_TRANSLATION_01,
BACON_TRANSLATION_AB
];
export const BACON_CLEARER_MAP = {
[BACON_TRANSLATION_01]: /[^01]/g,
[BACON_TRANSLATION_AB]: /[^ABab]/g,
[BACON_TRANSLATION_CASE]: /[^A-Za-z]/g,
};
export const BACON_NORMALIZE_MAP = {
[BACON_TRANSLATION_AB]: {
"A": "0",
"B": "1",
"a": "0",
"b": "1"
},
};
/**
* Swaps zeros to ones and ones to zeros.
*
* @param {string} data
* @returns {string}
*
* @example
* // returns "11001 01010"
* swapZeroAndOne("00110 10101");
*/
export function swapZeroAndOne(string) {
return string.replace(/[01]/g, function (c) {
return {
"0": "1",
"1": "0"
}[c];
});
}

0
src/core/lib/Base58.mjs Executable file → Normal file
View file

15
src/core/lib/Base64.mjs Executable file → Normal file
View file

@ -6,13 +6,13 @@
* @license Apache-2.0
*/
import Utils from "../Utils";
import Utils from "../Utils.mjs";
/**
* Base64's the input byte array using the given alphabet, returning a string.
*
* @param {byteArray|Uint8Array|string} data
* @param {byteArray|Uint8Array|ArrayBuffer|string} data
* @param {string} [alphabet="A-Za-z0-9+/="]
* @returns {string}
*
@ -25,6 +25,9 @@ import Utils from "../Utils";
*/
export function toBase64(data, alphabet="A-Za-z0-9+/=") {
if (!data) return "";
if (data instanceof ArrayBuffer) {
data = new Uint8Array(data);
}
if (typeof data == "string") {
data = Utils.strToByteArray(data);
}
@ -63,7 +66,7 @@ export function toBase64(data, alphabet="A-Za-z0-9+/=") {
/**
* UnBase64's the input string using the given alphabet, returning a byte array.
*
* @param {byteArray} data
* @param {string} data
* @param {string} [alphabet="A-Za-z0-9+/="]
* @param {string} [returnType="string"] - Either "string" or "byteArray"
* @param {boolean} [removeNonAlphChars=true]
@ -126,14 +129,14 @@ export function fromBase64(data, alphabet="A-Za-z0-9+/=", returnType="string", r
* Base64 alphabets.
*/
export const ALPHABET_OPTIONS = [
{name: "Standard: A-Za-z0-9+/=", value: "A-Za-z0-9+/="},
{name: "URL safe: A-Za-z0-9-_", value: "A-Za-z0-9-_"},
{name: "Standard (RFC 4648): A-Za-z0-9+/=", value: "A-Za-z0-9+/="},
{name: "URL safe (RFC 4648 \u00A75): A-Za-z0-9-_", value: "A-Za-z0-9-_"},
{name: "Filename safe: A-Za-z0-9+-=", value: "A-Za-z0-9+\\-="},
{name: "itoa64: ./0-9A-Za-z=", value: "./0-9A-Za-z="},
{name: "XML: A-Za-z0-9_.", value: "A-Za-z0-9_."},
{name: "y64: A-Za-z0-9._-", value: "A-Za-z0-9._-"},
{name: "z64: 0-9a-zA-Z+/=", value: "0-9a-zA-Z+/="},
{name: "Radix-64: 0-9A-Za-z+/=", value: "0-9A-Za-z+/="},
{name: "Radix-64 (RFC 4880): 0-9A-Za-z+/=", value: "0-9A-Za-z+/="},
{name: "Uuencoding: [space]-_", value: " -_"},
{name: "Xxencoding: +-0-9A-Za-z", value: "+\\-0-9A-Za-z"},
{name: "BinHex: !-,-0-689@A-NP-VX-Z[`a-fh-mp-r", value: "!-,-0-689@A-NP-VX-Z[`a-fh-mp-r"},

70
src/core/lib/Binary.mjs Normal file
View file

@ -0,0 +1,70 @@
/**
* Binary functions.
*
* @author n1474335 [n1474335@gmail.com]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
import Utils from "../Utils.mjs";
/**
* Convert a byte array into a binary string.
*
* @param {Uint8Array|byteArray} data
* @param {string} [delim="Space"]
* @param {number} [padding=8]
* @returns {string}
*
* @example
* // returns "00010000 00100000 00110000"
* toBinary([10,20,30]);
*
* // returns "00010000 00100000 00110000"
* toBinary([10,20,30], ":");
*/
export function toBinary(data, delim="Space", padding=8) {
if (!data) return "";
delim = Utils.charRep(delim);
let output = "";
for (let i = 0; i < data.length; i++) {
output += data[i].toString(2).padStart(padding, "0") + delim;
}
if (delim.length) {
return output.slice(0, -delim.length);
} else {
return output;
}
}
/**
* Convert a binary string into a byte array.
*
* @param {string} data
* @param {string} [delim]
* @param {number} [byteLen=8]
* @returns {byteArray}
*
* @example
* // returns [10,20,30]
* fromBinary("00010000 00100000 00110000");
*
* // returns [10,20,30]
* fromBinary("00010000:00100000:00110000", "Colon");
*/
export function fromBinary(data, delim="Space", byteLen=8) {
const delimRegex = Utils.regexRep(delim);
data = data.replace(delimRegex, "");
const output = [];
for (let i = 0; i < data.length; i += byteLen) {
output.push(parseInt(data.substr(i, byteLen), 2));
}
return output;
}

8
src/core/lib/BitwiseOp.mjs
View file

@ -9,7 +9,7 @@
/**
* Runs bitwise operations across the input data.
*
* @param {byteArray} input
* @param {byteArray|Uint8Array} input
* @param {byteArray} key
* @param {function} func - The bitwise calculation to carry out
* @param {boolean} nullPreserving
@ -116,3 +116,9 @@ export function sub(operand, key) {
const result = operand - key;
return (result < 0) ? 256 + result : result;
}
/**
* Delimiter options for bitwise operations
*/
export const BITWISE_OP_DELIMS = ["Hex", "Decimal", "Binary", "Base64", "UTF8", "Latin1"];

756
src/core/lib/Bombe.mjs Normal file
View file

@ -0,0 +1,756 @@
/**
* Emulation of the Bombe machine.
*
* @author s2224834
* @author The National Museum of Computing - Bombe Rebuild Project
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
import Utils from "../Utils.mjs";
import {Rotor, Plugboard, a2i, i2a} from "./Enigma.mjs";
/**
* Convenience/optimisation subclass of Rotor
*
* This allows creating multiple Rotors which share backing maps, to avoid repeatedly parsing the
* rotor spec strings and duplicating the maps in memory.
*/
class CopyRotor extends Rotor {
/**
* Return a copy of this Rotor.
* @returns {Object}
*/
copy() {
const clone = {
map: this.map,
revMap: this.revMap,
pos: this.pos,
step: this.step,
transform: this.transform,
revTransform: this.revTransform,
};
return clone;
}
}
/**
* Node in the menu graph
*
* A node represents a cipher/plaintext letter.
*/
class Node {
/**
* Node constructor.
* @param {number} letter - The plain/ciphertext letter this node represents (as a number).
*/
constructor(letter) {
this.letter = letter;
this.edges = new Set();
this.visited = false;
}
}
/**
* Edge in the menu graph
*
* An edge represents an Enigma machine transformation between two letters.
*/
class Edge {
/**
* Edge constructor - an Enigma machine mapping between letters
* @param {number} pos - The rotor position, relative to the beginning of the crib, at this edge
* @param {number} node1 - Letter at one end (as a number)
* @param {number} node2 - Letter at the other end
*/
constructor(pos, node1, node2) {
this.pos = pos;
this.node1 = node1;
this.node2 = node2;
node1.edges.add(this);
node2.edges.add(this);
this.visited = false;
}
/**
* Given the node at one end of this edge, return the other end.
* @param node {number} - The node we have
* @returns {number}
*/
getOther(node) {
return this.node1 === node ? this.node2 : this.node1;
}
}
/**
* As all the Bombe's rotors move in step, at any given point the vast majority of the scramblers
* in the machine share the majority of their state, which is hosted in this class.
*/
class SharedScrambler {
/**
* SharedScrambler constructor.
* @param {Object[]} rotors - List of rotors in the shared state _only_.
* @param {Object} reflector - The reflector in use.
*/
constructor(rotors, reflector) {
this.lowerCache = new Array(26);
this.higherCache = new Array(26);
for (let i=0; i<26; i++) {
this.higherCache[i] = new Array(26);
}
this.changeRotors(rotors, reflector);
}
/**
* Replace the rotors and reflector in this SharedScrambler.
* This takes care of flushing caches as well.
* @param {Object[]} rotors - List of rotors in the shared state _only_.
* @param {Object} reflector - The reflector in use.
*/
changeRotors(rotors, reflector) {
this.reflector = reflector;
this.rotors = rotors;
this.rotorsRev = [].concat(rotors).reverse();
this.cacheGen();
}
/**
* Step the rotors forward.
* @param {number} n - How many rotors to step. This includes the rotors which are not part of
* the shared state, so should be 2 or more.
*/
step(n) {
for (let i=0; i<n-1; i++) {
this.rotors[i].step();
}
this.cacheGen();
}
/**
* Optimisation: We pregenerate all routes through the machine with the top rotor removed,
* as these rarely change. This saves a lot of lookups. This function generates this route
* table.
* We also just-in-time cache the full routes through the scramblers, because after stepping
* the fast rotor some scramblers will be in states occupied by other scrambles on previous
* iterations.
*/
cacheGen() {
for (let i=0; i<26; i++) {
this.lowerCache[i] = undefined;
for (let j=0; j<26; j++) {
this.higherCache[i][j] = undefined;
}
}
for (let i=0; i<26; i++) {
if (this.lowerCache[i] !== undefined) {
continue;
}
let letter = i;
for (const rotor of this.rotors) {
letter = rotor.transform(letter);
}
letter = this.reflector.transform(letter);
for (const rotor of this.rotorsRev) {
letter = rotor.revTransform(letter);
}
// By symmetry
this.lowerCache[i] = letter;
this.lowerCache[letter] = i;
}
}
/**
* Map a letter through this (partial) scrambler.
* @param {number} i - The letter
* @returns {number}
*/
transform(i) {
return this.lowerCache[i];
}
}
/**
* Scrambler.
*
* This is effectively just an Enigma machine, but it only operates on one character at a time and
* the stepping mechanism is different.
*/
class Scrambler {
/** Scrambler constructor.
* @param {Object} base - The SharedScrambler whose state this scrambler uses
* @param {Object} rotor - The non-shared fast rotor in this scrambler
* @param {number} pos - Position offset from start of crib
* @param {number} end1 - Letter in menu this scrambler is attached to
* @param {number} end2 - Other letter in menu this scrambler is attached to
*/
constructor(base, rotor, pos, end1, end2) {
this.baseScrambler = base;
this.initialPos = pos;
this.changeRotor(rotor);
this.end1 = end1;
this.end2 = end2;
// For efficiency reasons, we pull the relevant shared cache from the baseScrambler into
// this object - this saves us a few pointer dereferences
this.cache = this.baseScrambler.higherCache[pos];
}
/**
* Replace the rotor in this scrambler.
* The position is reset automatically.
* @param {Object} rotor - New rotor
*/
changeRotor(rotor) {
this.rotor = rotor;
this.rotor.pos += this.initialPos;
}
/**
* Step the rotor forward.
*
* The base SharedScrambler needs to be instructed to step separately.
*/
step() {
// The Bombe steps the slowest rotor on an actual Enigma fastest, for reasons.
// ...but for optimisation reasons I'm going to cheat and not do that, as this vastly
// simplifies caching the state of the majority of the scramblers. The results are the
// same, just in a slightly different order.
this.rotor.step();
this.cache = this.baseScrambler.higherCache[this.rotor.pos];
}
/**
* Run a letter through the scrambler.
* @param {number} i - The letter to transform (as a number)
* @returns {number}
*/
transform(i) {
let letter = i;
const cached = this.cache[i];
if (cached !== undefined) {
return cached;
}
letter = this.rotor.transform(letter);
letter = this.baseScrambler.transform(letter);
letter = this.rotor.revTransform(letter);
this.cache[i] = letter;
this.cache[letter] = i;
return letter;
}
/**
* Given one letter in the menu this scrambler maps to, return the other.
* @param end {number} - The node we have
* @returns {number}
*/
getOtherEnd(end) {
return this.end1 === end ? this.end2 : this.end1;
}
/**
* Read the position this scrambler is set to.
* Note that because of Enigma's stepping, you need to set an actual Enigma to the previous
* position in order to get it to make a certain set of electrical connections when a button
* is pressed - this function *does* take this into account.
* However, as with the rest of the Bombe, it does not take stepping into account - the middle
* and slow rotors are treated as static.
* @return {string}
*/
getPos() {
let result = "";
// Roll back the fast rotor by one step
let pos = Utils.mod(this.rotor.pos - 1, 26);
result += i2a(pos);
for (let i=0; i<this.baseScrambler.rotors.length; i++) {
pos = this.baseScrambler.rotors[i].pos;
result += i2a(pos);
}
return result.split("").reverse().join("");
}
}
/**
* Bombe simulator class.
*/
export class BombeMachine {
/**
* Construct a Bombe.
*
* Note that there is no handling of offsets here: the crib specified must exactly match the
* ciphertext. It will check that the crib is sane (length is vaguely sensible and there's no
* matching characters between crib and ciphertext) but cannot check further - if it's wrong
* your results will be wrong!
*
* There is also no handling of rotor stepping - if the target Enigma stepped in the middle of
* your crib, you're out of luck. TODO: Allow specifying a step point - this is fairly easy to
* configure on a real Bombe, but we're not clear on whether it was ever actually done for
* real (there would almost certainly have been better ways of attacking in most situations
* than attempting to exhaust options for the stepping point, but in some circumstances, e.g.
* via Banburismus, the stepping point might have been known).
*
* @param {string[]} rotors - list of rotor spec strings (without step points!)
* @param {Object} reflector - Reflector object
* @param {string} ciphertext - The ciphertext to attack
* @param {string} crib - Known plaintext for this ciphertext
* @param {boolean} check - Whether to use the checking machine
* @param {function} update - Function to call to send status updates (optional)
*/
constructor(rotors, reflector, ciphertext, crib, check, update=undefined) {
if (ciphertext.length < crib.length) {
throw new OperationError("Crib overruns supplied ciphertext");
}
if (crib.length < 2) {
// This is the absolute bare minimum to be sane, and even then it's likely too short to
// be useful
throw new OperationError("Crib is too short");
}
if (crib.length > 25) {
// A crib longer than this will definitely cause the middle rotor to step somewhere
// A shorter crib is preferable to reduce this chance, of course
throw new OperationError("Crib is too long");
}
for (let i=0; i<crib.length; i++) {
if (ciphertext[i] === crib[i]) {
throw new OperationError(`Invalid crib: character ${ciphertext[i]} at pos ${i} in both ciphertext and crib`);
}
}
this.ciphertext = ciphertext;
this.crib = crib;
this.initRotors(rotors);
this.check = check;
this.updateFn = update;
const [mostConnected, edges] = this.makeMenu();
// This is the bundle of wires corresponding to the 26 letters within each of the 26
// possible nodes in the menu
this.wires = new Array(26*26);
// These are the pseudo-Engima devices corresponding to each edge in the menu, and the
// nodes in the menu they each connect to
this.scramblers = new Array();
for (let i=0; i<26; i++) {
this.scramblers.push(new Array());
}
this.sharedScrambler = new SharedScrambler(this.baseRotors.slice(1), reflector);
this.allScramblers = new Array();
this.indicator = undefined;
for (const edge of edges) {
const cRotor = this.baseRotors[0].copy();
const end1 = a2i(edge.node1.letter);
const end2 = a2i(edge.node2.letter);
const scrambler = new Scrambler(this.sharedScrambler, cRotor, edge.pos, end1, end2);
if (edge.pos === 0) {
this.indicator = scrambler;
}
this.scramblers[end1].push(scrambler);
this.scramblers[end2].push(scrambler);
this.allScramblers.push(scrambler);
}
// The Bombe uses a set of rotors to keep track of what settings it's testing. We cheat and
// use one of the actual scramblers if there's one in the right position, but if not we'll
// just create one.
if (this.indicator === undefined) {
this.indicator = new Scrambler(this.sharedScrambler, this.baseRotors[0].copy(), 0, undefined, undefined);
this.allScramblers.push(this.indicator);
}
this.testRegister = a2i(mostConnected.letter);
// This is an arbitrary letter other than the most connected letter
for (const edge of mostConnected.edges) {
this.testInput = [this.testRegister, a2i(edge.getOther(mostConnected).letter)];
break;
}
}
/**
* Build Rotor objects from list of rotor wiring strings.
* @param {string[]} rotors - List of rotor wiring strings
*/
initRotors(rotors) {
// This is ordered from the Enigma fast rotor to the slow, so bottom to top for the Bombe
this.baseRotors = [];
for (const rstr of rotors) {
const rotor = new CopyRotor(rstr, "", "A", "A");
this.baseRotors.push(rotor);
}
}
/**
* Replace the rotors and reflector in all components of this Bombe.
* @param {string[]} rotors - List of rotor wiring strings
* @param {Object} reflector - Reflector object
*/
changeRotors(rotors, reflector) {
// At the end of the run, the rotors are all back in the same position they started
this.initRotors(rotors);
this.sharedScrambler.changeRotors(this.baseRotors.slice(1), reflector);
for (const scrambler of this.allScramblers) {
scrambler.changeRotor(this.baseRotors[0].copy());
}
}
/**
* If we have a way of sending status messages, do so.
* @param {...*} msg - Message to send.
*/
update(...msg) {
if (this.updateFn !== undefined) {
this.updateFn(...msg);
}
}
/**
* Recursive depth-first search on the menu graph.
* This is used to a) isolate unconnected sub-graphs, and b) count the number of loops in each
* of those graphs.
* @param {Object} node - Node object to start the search from
* @returns {[number, number, Object, number, Object[]} - loop count, node count, most connected
* node, order of most connected node, list of edges in this sub-graph
*/
dfs(node) {
let loops = 0;
let nNodes = 1;
let mostConnected = node;
let nConnections = mostConnected.edges.size;
let edges = new Set();
node.visited = true;
for (const edge of node.edges) {
if (edge.visited) {
// Already been here from the other end.
continue;
}
edge.visited = true;
edges.add(edge);
const other = edge.getOther(node);
if (other.visited) {
// We have a loop, record that and continue
loops += 1;
continue;
}
// This is a newly visited node
const [oLoops, oNNodes, oMostConnected, oNConnections, oEdges] = this.dfs(other);
loops += oLoops;
nNodes += oNNodes;
edges = new Set([...edges, ...oEdges]);
if (oNConnections > nConnections) {
mostConnected = oMostConnected;
nConnections = oNConnections;
}
}
return [loops, nNodes, mostConnected, nConnections, edges];
}
/**
* Build a menu from the ciphertext and crib.
* A menu is just a graph where letters in either the ciphertext or crib (Enigma is symmetric,
* so there's no difference mathematically) are nodes and states of the Enigma machine itself
* are the edges.
* Additionally, we want a single connected graph, and of the subgraphs available, we want the
* one with the most loops (since these generate feedback cycles which efficiently close off
* disallowed states).
* Finally, we want to identify the most connected node in that graph (as it's the best choice
* of measurement point).
* @returns [Object, Object[]] - the most connected node, and the list of edges in the subgraph
*/
makeMenu() {
// First, we make a graph of all of the mappings given by the crib
// Make all nodes first
const nodes = new Map();
for (const c of this.ciphertext + this.crib) {
if (!nodes.has(c)) {
const node = new Node(c);
nodes.set(c, node);
}
}
// Then all edges
for (let i=0; i<this.crib.length; i++) {
const a = this.crib[i];
const b = this.ciphertext[i];
new Edge(i, nodes.get(a), nodes.get(b));
}
// list of [loop_count, node_count, most_connected_node, connections_on_most_connected, edges]
const graphs = [];
// Then, for each unconnected subgraph, we count the number of loops and nodes
for (const start of nodes.keys()) {
if (nodes.get(start).visited) {
continue;
}
const subgraph = this.dfs(nodes.get(start));
graphs.push(subgraph);
}
// Return the subgraph with the most loops (ties broken by node count)
graphs.sort((a, b) => {
let result = b[0] - a[0];
if (result === 0) {
result = b[1] - a[1];
}
return result;
});
this.nLoops = graphs[0][0];
return [graphs[0][2], graphs[0][4]];
}
/**
* Bombe electrical simulation. Energise a wire. For all connected wires (both via the diagonal
* board and via the scramblers), energise them too, recursively.
* @param {number} i - Bombe wire bundle
* @param {number} j - Bombe stecker hypothesis wire within bundle
*/
energise(i, j) {
const idx = 26*i + j;
if (this.wires[idx]) {
return;
}
this.wires[idx] = true;
// Welchman's diagonal board: if A steckers to B, that implies B steckers to A. Handle
// both.
const idxPair = 26*j + i;
this.wires[idxPair] = true;
if (i === this.testRegister || j === this.testRegister) {
this.energiseCount++;
if (this.energiseCount === 26) {
// no point continuing, bail out
return;
}
}
for (let k=0; k<this.scramblers[i].length; k++) {
const scrambler = this.scramblers[i][k];
const out = scrambler.transform(j);
const other = scrambler.getOtherEnd(i);
// Lift the pre-check before the call, to save some function call overhead
const otherIdx = 26*other + out;
if (!this.wires[otherIdx]) {
this.energise(other, out);
if (this.energiseCount === 26) {
return;
}
}
}
if (i === j) {
return;
}
for (let k=0; k<this.scramblers[j].length; k++) {
const scrambler = this.scramblers[j][k];
const out = scrambler.transform(i);
const other = scrambler.getOtherEnd(j);
const otherIdx = 26*other + out;
if (!this.wires[otherIdx]) {
this.energise(other, out);
if (this.energiseCount === 26) {
return;
}
}
}
}
/**
* Trial decryption at the current setting.
* Used after we get a stop.
* This applies the detected stecker pair if we have one. It does not handle the other
* steckering or stepping (which is why we limit it to 26 characters, since it's guaranteed to
* be wrong after that anyway).
* @param {string} stecker - Known stecker spec string.
* @returns {string}
*/
tryDecrypt(stecker) {
const fastRotor = this.indicator.rotor;
const initialPos = fastRotor.pos;
const res = [];
const plugboard = new Plugboard(stecker);
// The indicator scrambler starts in the right place for the beginning of the ciphertext.
for (let i=0; i<Math.min(26, this.ciphertext.length); i++) {
const t = this.indicator.transform(plugboard.transform(a2i(this.ciphertext[i])));
res.push(i2a(plugboard.transform(t)));
this.indicator.step(1);
}
fastRotor.pos = initialPos;
return res.join("");
}
/**
* Format a steckered pair, in sorted order to allow uniquing.
* @param {number} a - A letter
* @param {number} b - Its stecker pair
* @returns {string}
*/
formatPair(a, b) {
if (a < b) {
return `${i2a(a)}${i2a(b)}`;
}
return `${i2a(b)}${i2a(a)}`;
}
/**
* The checking machine was used to manually verify Bombe stops. Using a device which was
* effectively a non-stepping Enigma, the user would walk through each of the links in the
* menu at the rotor positions determined by the Bombe. By starting with the stecker pair the
* Bombe gives us, we find the stecker pair of each connected letter in the graph, and so on.
* If a contradiction is reached, the stop is invalid. If not, we have most (but not
* necessarily all) of the plugboard connections.
* You will notice that this procedure is exactly the same as what the Bombe itself does, only
* we start with an assumed good hypothesis and read out the stecker pair for every letter.
* On the real hardware that wasn't practical, but fortunately we're not the real hardware, so
* we don't need to implement the manual checking machine procedure.
* @param {number} pair - The stecker pair of the test register.
* @returns {string} - The empty string for invalid stops, or a plugboard configuration string
* containing all known pairs.
*/
checkingMachine(pair) {
if (pair !== this.testInput[1]) {
// We have a new hypothesis for this stop - apply the new one.
// De-energise the board
for (let i=0; i<this.wires.length; i++) {
this.wires[i] = false;
}
this.energiseCount = 0;
// Re-energise with the corrected hypothesis
this.energise(this.testRegister, pair);
}
const results = new Set();
results.add(this.formatPair(this.testRegister, pair));
for (let i=0; i<26; i++) {
let count = 0;
let other;
for (let j=0; j<26; j++) {
if (this.wires[i*26 + j]) {
count++;
other = j;
}
}
if (count > 1) {
// This is an invalid stop.
return "";
} else if (count === 0) {
// No information about steckering from this wire
continue;
}
results.add(this.formatPair(i, other));
}
return [...results].join(" ");
}
/**
* Check to see if the Bombe has stopped. If so, process the stop.
* @returns {(undefined|string[3])} - Undefined for no stop, or [rotor settings, plugboard settings, decryption preview]
*/
checkStop() {
// Count the energised outputs
const count = this.energiseCount;
if (count === 26) {
return undefined;
}
// If it's not all of them, we have a stop
let steckerPair;
// The Bombe tells us one stecker pair as well. The input wire and test register we
// started with are hypothesised to be a stecker pair.
if (count === 25) {
// Our steckering hypothesis is wrong. Correct value is the un-energised wire.
for (let j=0; j<26; j++) {
if (!this.wires[26*this.testRegister + j]) {
steckerPair = j;
break;
}
}
} else if (count === 1) {
// This means our hypothesis for the steckering is correct.
steckerPair = this.testInput[1];
} else {
// This was known as a "boxing stop" - we have a stop but not a single hypothesis.
// If this happens a lot it implies the menu isn't good enough.
// If we have the checking machine enabled, we're going to just check each wire in
// turn. If we get 0 or 1 hit, great.
// If we get multiple hits, or the checking machine is off, the user will just have to
// deal with it.
if (!this.check) {
// We can't draw any conclusions about the steckering (one could maybe suggest
// options in some cases, but too hard to present clearly).
return [this.indicator.getPos(), "??", this.tryDecrypt("")];
}
let stecker = undefined;
for (let i = 0; i < 26; i++) {
const newStecker = this.checkingMachine(i);
if (newStecker !== "") {
if (stecker !== undefined) {
// Multiple hypotheses can't be ruled out.
return [this.indicator.getPos(), "??", this.tryDecrypt("")];
}
stecker = newStecker;
}
}
if (stecker === undefined) {
// Checking machine ruled all possibilities out.
return undefined;
}
// If we got here, there was just one possibility allowed by the checking machine. Success.
return [this.indicator.getPos(), stecker, this.tryDecrypt(stecker)];
}
let stecker;
if (this.check) {
stecker = this.checkingMachine(steckerPair);
if (stecker === "") {
// Invalid stop - don't count it, don't return it
return undefined;
}
} else {
stecker = `${i2a(this.testRegister)}${i2a(steckerPair)}`;
}
const testDecrypt = this.tryDecrypt(stecker);
return [this.indicator.getPos(), stecker, testDecrypt];
}
/**
* Having set up the Bombe, do the actual attack run. This tries every possible rotor setting
* and attempts to logically invalidate them. If it can't, it's added to the list of candidate
* solutions.
* @returns {string[][3]} - list of 3-tuples of candidate rotor setting, plugboard settings, and decryption preview
*/
run() {
let stops = 0;
const result = [];
// For each possible rotor setting
const nChecks = Math.pow(26, this.baseRotors.length);
for (let i=1; i<=nChecks; i++) {
// Benchmarking suggests this is faster than using .fill()
for (let i=0; i<this.wires.length; i++) {
this.wires[i] = false;
}
this.energiseCount = 0;
// Energise the test input, follow the current through each scrambler
// (and the diagonal board)
this.energise(...this.testInput);
const stop = this.checkStop();
if (stop !== undefined) {
stops++;
result.push(stop);
}
// Step all the scramblers
// This loop counts how many rotors have reached their starting position (meaning the
// next one needs to step as well)
let n = 1;
for (let j=1; j<this.baseRotors.length; j++) {
if ((i % Math.pow(26, j)) === 0) {
n++;
} else {
break;
}
}
if (n > 1) {
this.sharedScrambler.step(n);
}
for (const scrambler of this.allScramblers) {
scrambler.step();
}
// Send status messages at what seems to be a reasonably sensible frequency
// (note this won't be triggered on 3-rotor runs - they run fast enough it doesn't seem necessary)
if (n > 3) {
this.update(this.nLoops, stops, i/nChecks);
}
}
return result;
}
}

0
src/core/lib/CanvasComponents.mjs Executable file → Normal file
View file

178
src/core/lib/Charts.mjs Normal file
View file

@ -0,0 +1,178 @@
/**
* @author tlwr [toby@toby.codes]
* @author Matt C [me@mitt.dev]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
/**
* @constant
* @default
*/
export const RECORD_DELIMITER_OPTIONS = ["Line feed", "CRLF"];
/**
* @constant
* @default
*/
export const FIELD_DELIMITER_OPTIONS = ["Space", "Comma", "Semi-colon", "Colon", "Tab"];
/**
* Default from colour
*
* @constant
* @default
*/
export const COLOURS = {
min: "white",
max: "black"
};
/**
* Gets values from input for a plot.
*
* @param {string} input
* @param {string} recordDelimiter
* @param {string} fieldDelimiter
* @param {boolean} columnHeadingsAreIncluded - whether we should skip the first record
* @param {number} length
* @returns {Object[]}
*/
export function getValues(input, recordDelimiter, fieldDelimiter, columnHeadingsAreIncluded, length) {
let headings;
const values = [];
input
.split(recordDelimiter)
.forEach((row, rowIndex) => {
const split = row.split(fieldDelimiter);
if (split.length !== length) throw new OperationError(`Each row must have length ${length}.`);
if (columnHeadingsAreIncluded && rowIndex === 0) {
headings = split;
} else {
values.push(split);
}
});
return { headings, values };
}
/**
* Gets values from input for a scatter plot.
*
* @param {string} input
* @param {string} recordDelimiter
* @param {string} fieldDelimiter
* @param {boolean} columnHeadingsAreIncluded - whether we should skip the first record
* @returns {Object[]}
*/
export function getScatterValues(input, recordDelimiter, fieldDelimiter, columnHeadingsAreIncluded) {
let { headings, values } = getValues(
input,
recordDelimiter,
fieldDelimiter,
columnHeadingsAreIncluded,
2
);
if (headings) {
headings = {x: headings[0], y: headings[1]};
}
values = values.map(row => {
const x = parseFloat(row[0], 10),
y = parseFloat(row[1], 10);
if (Number.isNaN(x)) throw new OperationError("Values must be numbers in base 10.");
if (Number.isNaN(y)) throw new OperationError("Values must be numbers in base 10.");
return [x, y];
});
return { headings, values };
}
/**
* Gets values from input for a scatter plot with colour from the third column.
*
* @param {string} input
* @param {string} recordDelimiter
* @param {string} fieldDelimiter
* @param {boolean} columnHeadingsAreIncluded - whether we should skip the first record
* @returns {Object[]}
*/
export function getScatterValuesWithColour(input, recordDelimiter, fieldDelimiter, columnHeadingsAreIncluded) {
let { headings, values } = getValues(
input,
recordDelimiter, fieldDelimiter,
columnHeadingsAreIncluded,
3
);
if (headings) {
headings = {x: headings[0], y: headings[1]};
}
values = values.map(row => {
const x = parseFloat(row[0], 10),
y = parseFloat(row[1], 10),
colour = row[2];
if (Number.isNaN(x)) throw new OperationError("Values must be numbers in base 10.");
if (Number.isNaN(y)) throw new OperationError("Values must be numbers in base 10.");
return [x, y, colour];
});
return { headings, values };
}
/**
* Gets values from input for a time series plot.
*
* @param {string} input
* @param {string} recordDelimiter
* @param {string} fieldDelimiter
* @param {boolean} columnHeadingsAreIncluded - whether we should skip the first record
* @returns {Object[]}
*/
export function getSeriesValues(input, recordDelimiter, fieldDelimiter, columnHeadingsAreIncluded) {
const { values } = getValues(
input,
recordDelimiter, fieldDelimiter,
false,
3
);
let xValues = new Set();
const series = {};
values.forEach(row => {
const serie = row[0],
xVal = row[1],
val = parseFloat(row[2], 10);
if (Number.isNaN(val)) throw new OperationError("Values must be numbers in base 10.");
xValues.add(xVal);
if (typeof series[serie] === "undefined") series[serie] = {};
series[serie][xVal] = val;
});
xValues = new Array(...xValues);
const seriesList = [];
for (const seriesName in series) {
const serie = series[seriesName];
seriesList.push({name: seriesName, data: serie});
}
return { xValues, series: seriesList };
}

2
src/core/lib/Ciphers.mjs
View file

@ -9,7 +9,7 @@
*
*/
import OperationError from "../errors/OperationError";
import OperationError from "../errors/OperationError.mjs";
import CryptoJS from "crypto-js";
/**

668
src/core/lib/ConvertCoordinates.mjs Normal file
View file

@ -0,0 +1,668 @@
/**
* Co-ordinate conversion resources.
*
* @author j433866 [j433866@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
import geohash from "ngeohash";
/*
Currently unable to update to geodesy v2 as we cannot load .js modules into a .mjs file.
When we do update, imports will look like this:
import LatLonEllipsoidal from "geodesy/latlon-ellipsoidal.js";
import Mgrs from "geodesy/mgrs.js";
import OsGridRef from "geodesy/osgridref.js";
import Utm from "geodesy/utm.js";
*/
import geodesy from "geodesy";
const LatLonEllipsoidal = geodesy.LatLonEllipsoidal,
Mgrs = geodesy.Mgrs,
OsGridRef = geodesy.OsGridRef,
Utm = geodesy.Utm;
/**
* Co-ordinate formats
*/
export const FORMATS = [
"Degrees Minutes Seconds",
"Degrees Decimal Minutes",
"Decimal Degrees",
"Geohash",
"Military Grid Reference System",
"Ordnance Survey National Grid",
"Universal Transverse Mercator"
];
/**
* Formats that should be passed to the conversion module as-is
*/
const NO_CHANGE = [
"Geohash",
"Military Grid Reference System",
"Ordnance Survey National Grid",
"Universal Transverse Mercator",
];
/**
* Convert a given latitude and longitude into a different format.
*
* @param {string} input - Input string to be converted
* @param {string} inFormat - Format of the input coordinates
* @param {string} inDelim - The delimiter splitting the lat/long of the input
* @param {string} outFormat - Format to convert to
* @param {string} outDelim - The delimiter to separate the output with
* @param {string} includeDir - Whether or not to include the compass direction in the output
* @param {number} precision - Precision of the result
* @returns {string} A formatted string of the converted co-ordinates
*/
export function convertCoordinates (input, inFormat, inDelim, outFormat, outDelim, includeDir, precision) {
let isPair = false,
split,
latlon,
convLat,
convLon,
conv,
hash,
utm,
mgrs,
osng,
splitLat,
splitLong,
lat,
lon;
// Can't have a precision less than 0!
if (precision < 0) {
precision = 0;
}
if (inDelim === "Auto") {
// Try to detect a delimiter in the input.
inDelim = findDelim(input);
if (inDelim === null) {
throw new OperationError("Unable to detect the input delimiter automatically.");
}
} else if (!inDelim.includes("Direction")) {
// Convert the delimiter argument value to the actual character
inDelim = realDelim(inDelim);
}
if (inFormat === "Auto") {
// Try to detect the format of the input data
inFormat = findFormat(input, inDelim);
if (inFormat === null) {
throw new OperationError("Unable to detect the input format automatically.");
}
}
// Convert the output delimiter argument to the real character
outDelim = realDelim(outDelim);
if (!NO_CHANGE.includes(inFormat)) {
if (inDelim.includes("Direction")) {
// Split on directions
split = input.split(/[NnEeSsWw]/g);
if (split[0] === "") {
// Remove first element if direction preceding
split = split.slice(1);
}
} else {
split = input.split(inDelim);
}
// Replace any co-ordinate symbols with spaces so we can split on them later
for (let i = 0; i < split.length; i++) {
split[i] = split[i].replace(/[°˝´'"]/g, " ");
}
if (split.length > 1) {
isPair = true;
}
} else {
// Remove any delimiters from the input
input = input.replace(inDelim, "");
isPair = true;
}
// Conversions from the input format into a geodesy latlon object
switch (inFormat) {
case "Geohash":
hash = geohash.decode(input.replace(/[^A-Za-z0-9]/g, ""));
latlon = new LatLonEllipsoidal(hash.latitude, hash.longitude);
break;
case "Military Grid Reference System":
utm = Mgrs.parse(input.replace(/[^A-Za-z0-9]/g, "")).toUtm();
latlon = utm.toLatLonE();
break;
case "Ordnance Survey National Grid":
osng = OsGridRef.parse(input.replace(/[^A-Za-z0-9]/g, ""));
latlon = OsGridRef.osGridToLatLon(osng);
break;
case "Universal Transverse Mercator":
// Geodesy needs a space between the first 2 digits and the next letter
if (/^[\d]{2}[A-Za-z]/.test(input)) {
input = input.slice(0, 2) + " " + input.slice(2);
}
utm = Utm.parse(input);
latlon = utm.toLatLonE();
break;
case "Degrees Minutes Seconds":
if (isPair) {
// Split up the lat/long into degrees / minutes / seconds values
splitLat = splitInput(split[0]);
splitLong = splitInput(split[1]);
if (splitLat.length >= 3 && splitLong.length >= 3) {
lat = convDMSToDD(splitLat[0], splitLat[1], splitLat[2], 10);
lon = convDMSToDD(splitLong[0], splitLong[1], splitLong[2], 10);
latlon = new LatLonEllipsoidal(lat.degrees, lon.degrees);
} else {
throw new OperationError("Invalid co-ordinate format for Degrees Minutes Seconds");
}
} else {
// Not a pair, so only try to convert one set of co-ordinates
splitLat = splitInput(split[0]);
if (splitLat.length >= 3) {
lat = convDMSToDD(splitLat[0], splitLat[1], splitLat[2]);
latlon = new LatLonEllipsoidal(lat.degrees, lat.degrees);
} else {
throw new OperationError("Invalid co-ordinate format for Degrees Minutes Seconds");
}
}
break;
case "Degrees Decimal Minutes":
if (isPair) {
splitLat = splitInput(split[0]);
splitLong = splitInput(split[1]);
if (splitLat.length !== 2 || splitLong.length !== 2) {
throw new OperationError("Invalid co-ordinate format for Degrees Decimal Minutes.");
}
// Convert to decimal degrees, and then convert to a geodesy object
lat = convDDMToDD(splitLat[0], splitLat[1], 10);
lon = convDDMToDD(splitLong[0], splitLong[1], 10);
latlon = new LatLonEllipsoidal(lat.degrees, lon.degrees);
} else {
// Not a pair, so only try to convert one set of co-ordinates
splitLat = splitInput(input);
if (splitLat.length !== 2) {
throw new OperationError("Invalid co-ordinate format for Degrees Decimal Minutes.");
}
lat = convDDMToDD(splitLat[0], splitLat[1], 10);
latlon = new LatLonEllipsoidal(lat.degrees, lat.degrees);
}
break;
case "Decimal Degrees":
if (isPair) {
splitLat = splitInput(split[0]);
splitLong = splitInput(split[1]);
if (splitLat.length !== 1 || splitLong.length !== 1) {
throw new OperationError("Invalid co-ordinate format for Decimal Degrees.");
}
latlon = new LatLonEllipsoidal(splitLat[0], splitLong[0]);
} else {
// Not a pair, so only try to convert one set of co-ordinates
splitLat = splitInput(split[0]);
if (splitLat.length !== 1) {
throw new OperationError("Invalid co-ordinate format for Decimal Degrees.");
}
latlon = new LatLonEllipsoidal(splitLat[0], splitLat[0]);
}
break;
default:
throw new OperationError(`Unknown input format '${inFormat}'`);
}
// Everything is now a geodesy latlon object
// These store the latitude and longitude as decimal
if (inFormat.includes("Degrees")) {
// If the input string contains directions, we need to check if they're S or W.
// If either of the directions are, we should make the decimal value negative
const dirs = input.toUpperCase().match(/[NESW]/g);
if (dirs && dirs.length >= 1) {
// Make positive lat/lon values with S/W directions into negative values
if (dirs[0] === "S" || dirs[0] === "W" && latlon.lat > 0) {
latlon.lat = -latlon.lat;
}
if (dirs.length >= 2) {
if (dirs[1] === "S" || dirs[1] === "W" && latlon.lon > 0) {
latlon.lon = -latlon.lon;
}
}
}
}
// Try to find the compass directions of the lat and long
const [latDir, longDir] = findDirs(latlon.lat + "," + latlon.lon, ",");
// Output conversions for each output format
switch (outFormat) {
case "Decimal Degrees":
// We could use the built in latlon.toString(),
// but this makes adjusting the output harder
lat = convDDToDD(latlon.lat, precision);
lon = convDDToDD(latlon.lon, precision);
convLat = lat.string;
convLon = lon.string;
break;
case "Degrees Decimal Minutes":
lat = convDDToDDM(latlon.lat, precision);
lon = convDDToDDM(latlon.lon, precision);
convLat = lat.string;
convLon = lon.string;
break;
case "Degrees Minutes Seconds":
lat = convDDToDMS(latlon.lat, precision);
lon = convDDToDMS(latlon.lon, precision);
convLat = lat.string;
convLon = lon.string;
break;
case "Geohash":
convLat = geohash.encode(latlon.lat, latlon.lon, precision);
break;
case "Military Grid Reference System":
utm = latlon.toUtm();
mgrs = utm.toMgrs();
// MGRS wants a precision that's an even number between 2 and 10
if (precision % 2 !== 0) {
precision = precision + 1;
}
if (precision > 10) {
precision = 10;
}
convLat = mgrs.toString(precision);
break;
case "Ordnance Survey National Grid":
osng = OsGridRef.latLonToOsGrid(latlon);
if (osng.toString() === "") {
throw new OperationError("Could not convert co-ordinates to OS National Grid. Are the co-ordinates in range?");
}
// OSNG wants a precision that's an even number between 2 and 10
if (precision % 2 !== 0) {
precision = precision + 1;
}
if (precision > 10) {
precision = 10;
}
convLat = osng.toString(precision);
break;
case "Universal Transverse Mercator":
utm = latlon.toUtm();
convLat = utm.toString(precision);
break;
}
if (convLat === undefined) {
throw new OperationError("Error converting co-ordinates.");
}
if (outFormat.includes("Degrees")) {
// Format DD/DDM/DMS for output
// If we're outputting a compass direction, remove the negative sign
if (latDir === "S" && includeDir !== "None") {
convLat = convLat.replace("-", "");
}
if (longDir === "W" && includeDir !== "None") {
convLon = convLon.replace("-", "");
}
let outConv = "";
if (includeDir === "Before") {
outConv += latDir + " ";
}
outConv += convLat;
if (includeDir === "After") {
outConv += " " + latDir;
}
outConv += outDelim;
if (isPair) {
if (includeDir === "Before") {
outConv += longDir + " ";
}
outConv += convLon;
if (includeDir === "After") {
outConv += " " + longDir;
}
outConv += outDelim;
}
conv = outConv;
} else {
conv = convLat + outDelim;
}
return conv;
}
/**
* Split up the input using a space or degrees signs, and sanitise the result
*
* @param {string} input - The input data to be split
* @returns {number[]} An array of the different items in the string, stored as floats
*/
function splitInput (input) {
const split = [];
input.split(/\s+/).forEach(item => {
// Remove any character that isn't a digit, decimal point or negative sign
item = item.replace(/[^0-9.-]/g, "");
if (item.length > 0) {
// Turn the item into a float
split.push(parseFloat(item));
}
});
return split;
}
/**
* Convert Degrees Minutes Seconds to Decimal Degrees
*
* @param {number} degrees - The degrees of the input co-ordinates
* @param {number} minutes - The minutes of the input co-ordinates
* @param {number} seconds - The seconds of the input co-ordinates
* @param {number} precision - The precision the result should be rounded to
* @returns {{string: string, degrees: number}} An object containing the raw converted value (obj.degrees), and a formatted string version (obj.string)
*/
function convDMSToDD (degrees, minutes, seconds, precision) {
const absDegrees = Math.abs(degrees);
let conv = absDegrees + (minutes / 60) + (seconds / 3600);
let outString = round(conv, precision) + "°";
if (isNegativeZero(degrees) || degrees < 0) {
conv = -conv;
outString = "-" + outString;
}
return {
"degrees": conv,
"string": outString
};
}
/**
* Convert Decimal Degrees Minutes to Decimal Degrees
*
* @param {number} degrees - The input degrees to be converted
* @param {number} minutes - The input minutes to be converted
* @param {number} precision - The precision which the result should be rounded to
* @returns {{string: string, degrees: number}} An object containing the raw converted value (obj.degrees), and a formatted string version (obj.string)
*/
function convDDMToDD (degrees, minutes, precision) {
const absDegrees = Math.abs(degrees);
let conv = absDegrees + minutes / 60;
let outString = round(conv, precision) + "°";
if (isNegativeZero(degrees) || degrees < 0) {
conv = -conv;
outString = "-" + outString;
}
return {
"degrees": conv,
"string": outString
};
}
/**
* Convert Decimal Degrees to Decimal Degrees
*
* Doesn't affect the input, just puts it into an object
* @param {number} degrees - The input degrees to be converted
* @param {number} precision - The precision which the result should be rounded to
* @returns {{string: string, degrees: number}} An object containing the raw converted value (obj.degrees), and a formatted string version (obj.string)
*/
function convDDToDD (degrees, precision) {
return {
"degrees": degrees,
"string": round(degrees, precision) + "°"
};
}
/**
* Convert Decimal Degrees to Degrees Minutes Seconds
*
* @param {number} decDegrees - The input data to be converted
* @param {number} precision - The precision which the result should be rounded to
* @returns {{string: string, degrees: number, minutes: number, seconds: number}} An object containing the raw converted value as separate numbers (.degrees, .minutes, .seconds), and a formatted string version (obj.string)
*/
function convDDToDMS (decDegrees, precision) {
const absDegrees = Math.abs(decDegrees);
let degrees = Math.floor(absDegrees);
const minutes = Math.floor(60 * (absDegrees - degrees)),
seconds = round(3600 * (absDegrees - degrees) - 60 * minutes, precision);
let outString = degrees + "° " + minutes + "' " + seconds + "\"";
if (isNegativeZero(decDegrees) || decDegrees < 0) {
degrees = -degrees;
outString = "-" + outString;
}
return {
"degrees": degrees,
"minutes": minutes,
"seconds": seconds,
"string": outString
};
}
/**
* Convert Decimal Degrees to Degrees Decimal Minutes
*
* @param {number} decDegrees - The input degrees to be converted
* @param {number} precision - The precision the input data should be rounded to
* @returns {{string: string, degrees: number, minutes: number}} An object containing the raw converted value as separate numbers (.degrees, .minutes), and a formatted string version (obj.string)
*/
function convDDToDDM (decDegrees, precision) {
const absDegrees = Math.abs(decDegrees);
let degrees = Math.floor(absDegrees);
const minutes = absDegrees - degrees,
decMinutes = round(minutes * 60, precision);
let outString = degrees + "° " + decMinutes + "'";
if (decDegrees < 0 || isNegativeZero(decDegrees)) {
degrees = -degrees;
outString = "-" + outString;
}
return {
"degrees": degrees,
"minutes": decMinutes,
"string": outString,
};
}
/**
* Finds and returns the compass directions in an input string
*
* @param {string} input - The input co-ordinates containing the direction
* @param {string} delim - The delimiter separating latitide and longitude
* @returns {string[]} String array containing the latitude and longitude directions
*/
export function findDirs(input, delim) {
const upperInput = input.toUpperCase();
const dirExp = new RegExp(/[NESW]/g);
const dirs = upperInput.match(dirExp);
if (dirs) {
// If there's actually compass directions
// in the input, use these to work out the direction
if (dirs.length <= 2 && dirs.length >= 1) {
return dirs.length === 2 ? [dirs[0], dirs[1]] : [dirs[0], ""];
}
}
// Nothing was returned, so guess the directions
let lat = upperInput,
long,
latDir = "",
longDir = "";
if (!delim.includes("Direction")) {
if (upperInput.includes(delim)) {
const split = upperInput.split(delim);
if (split.length >= 1) {
if (split[0] !== "") {
lat = split[0];
}
if (split.length >= 2 && split[1] !== "") {
long = split[1];
}
}
}
} else {
const split = upperInput.split(dirExp);
if (split.length > 1) {
lat = split[0] === "" ? split[1] : split[0];
if (split.length > 2 && split[2] !== "") {
long = split[2];
}
}
}
if (lat) {
lat = parseFloat(lat);
latDir = lat < 0 ? "S" : "N";
}
if (long) {
long = parseFloat(long);
longDir = long < 0 ? "W" : "E";
}
return [latDir, longDir];
}
/**
* Detects the co-ordinate format of the input data
*
* @param {string} input - The input data whose format we need to detect
* @param {string} delim - The delimiter separating the data in input
* @returns {string} The input format
*/
export function findFormat (input, delim) {
let testData;
const mgrsPattern = new RegExp(/^[0-9]{2}\s?[C-HJ-NP-X]{1}\s?[A-HJ-NP-Z][A-HJ-NP-V]\s?[0-9\s]+/),
osngPattern = new RegExp(/^[A-HJ-Z]{2}\s+[0-9\s]+$/),
geohashPattern = new RegExp(/^[0123456789BCDEFGHJKMNPQRSTUVWXYZ]+$/),
utmPattern = new RegExp(/^[0-9]{2}\s?[C-HJ-NP-X]\s[0-9.]+\s?[0-9.]+$/),
degPattern = new RegExp(/[°'"]/g);
input = input.trim();
if (delim !== null && delim.includes("Direction")) {
const split = input.split(/[NnEeSsWw]/);
if (split.length > 1) {
testData = split[0] === "" ? split[1] : split[0];
}
} else if (delim !== null && delim !== "") {
if (input.includes(delim)) {
const split = input.split(delim);
if (split.length > 1) {
testData = split[0] === "" ? split[1] : split[0];
}
} else {
testData = input;
}
}
// Test non-degrees formats
if (!degPattern.test(input)) {
const filteredInput = input.toUpperCase().replace(delim, "");
if (utmPattern.test(filteredInput)) {
return "Universal Transverse Mercator";
}
if (mgrsPattern.test(filteredInput)) {
return "Military Grid Reference System";
}
if (osngPattern.test(filteredInput)) {
return "Ordnance Survey National Grid";
}
if (geohashPattern.test(filteredInput)) {
return "Geohash";
}
}
// Test DMS/DDM/DD formats
if (testData !== undefined) {
const split = splitInput(testData);
switch (split.length) {
case 3:
return "Degrees Minutes Seconds";
case 2:
return "Degrees Decimal Minutes";
case 1:
return "Decimal Degrees";
}
}
return null;
}
/**
* Automatically find the delimeter type from the given input
*
* @param {string} input
* @returns {string} Delimiter type
*/
export function findDelim (input) {
input = input.trim();
const delims = [",", ";", ":"];
const testDir = input.match(/[NnEeSsWw]/g);
if (testDir !== null && testDir.length > 0 && testDir.length < 3) {
// Possibly contains a direction
const splitInput = input.split(/[NnEeSsWw]/);
if (splitInput.length <= 3 && splitInput.length > 0) {
// If there's 3 splits (one should be empty), then assume we have directions
if (splitInput[0] === "") {
return "Direction Preceding";
} else if (splitInput[splitInput.length - 1] === "") {
return "Direction Following";
}
}
}
// Loop through the standard delimiters, and try to find them in the input
for (let i = 0; i < delims.length; i++) {
const delim = delims[i];
if (input.includes(delim)) {
const splitInput = input.split(delim);
if (splitInput.length <= 3 && splitInput.length > 0) {
// Don't want to try and convert more than 2 co-ordinates
return delim;
}
}
}
return null;
}
/**
* Gets the real string for a delimiter name.
*
* @param {string} delim The delimiter to be matched
* @returns {string}
*/
export function realDelim (delim) {
return {
"Auto": "Auto",
"Space": " ",
"\\n": "\n",
"Comma": ",",
"Semi-colon": ";",
"Colon": ":"
}[delim];
}
/**
* Returns true if a zero is negative
*
* @param {number} zero
* @returns {boolean}
*/
function isNegativeZero(zero) {
return zero === 0 && (1/zero < 0);
}
/**
* Rounds a number to a specified number of decimal places
*
* @param {number} input - The number to be rounded
* @param {precision} precision - The number of decimal places the number should be rounded to
* @returns {number}
*/
function round(input, precision) {
precision = Math.pow(10, precision);
return Math.round(input * precision) / precision;
}

2
src/core/lib/Decimal.mjs
View file

@ -6,7 +6,7 @@
* @license Apache-2.0
*/
import Utils from "../Utils";
import Utils from "../Utils.mjs";
/**

9
src/core/lib/Delim.mjs
View file

@ -72,3 +72,12 @@ export const JOIN_DELIM_OPTIONS = [
{name: "Nothing (join chars)", value: ""}
];
/**
* RGBA list delimiters.
*/
export const RGBA_DELIM_OPTIONS = [
{name: "Comma", value: ","},
{name: "Space", value: " "},
{name: "CRLF", value: "\\r\\n"},
{name: "Line Feed", value: "\n"}
];

369
src/core/lib/Enigma.mjs Normal file
View file

@ -0,0 +1,369 @@
/**
* Emulation of the Enigma machine.
*
* @author s2224834
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
import Utils from "../Utils.mjs";
/**
* Provided default Enigma rotor set.
* These are specified as a list of mappings from the letters A through Z in order, optionally
* followed by < and a list of letters at which the rotor steps.
*/
export const ROTORS = [
{name: "I", value: "EKMFLGDQVZNTOWYHXUSPAIBRCJ<R"},
{name: "II", value: "AJDKSIRUXBLHWTMCQGZNPYFVOE<F"},
{name: "III", value: "BDFHJLCPRTXVZNYEIWGAKMUSQO<W"},
{name: "IV", value: "ESOVPZJAYQUIRHXLNFTGKDCMWB<K"},
{name: "V", value: "VZBRGITYUPSDNHLXAWMJQOFECK<A"},
{name: "VI", value: "JPGVOUMFYQBENHZRDKASXLICTW<AN"},
{name: "VII", value: "NZJHGRCXMYSWBOUFAIVLPEKQDT<AN"},
{name: "VIII", value: "FKQHTLXOCBJSPDZRAMEWNIUYGV<AN"},
];
export const ROTORS_FOURTH = [
{name: "Beta", value: "LEYJVCNIXWPBQMDRTAKZGFUHOS"},
{name: "Gamma", value: "FSOKANUERHMBTIYCWLQPZXVGJD"},
];
/**
* Provided default Enigma reflector set.
* These are specified as 13 space-separated transposed pairs covering every letter.
*/
export const REFLECTORS = [
{name: "B", value: "AY BR CU DH EQ FS GL IP JX KN MO TZ VW"},
{name: "C", value: "AF BV CP DJ EI GO HY KR LZ MX NW TQ SU"},
{name: "B Thin", value: "AE BN CK DQ FU GY HW IJ LO MP RX SZ TV"},
{name: "C Thin", value: "AR BD CO EJ FN GT HK IV LM PW QZ SX UY"},
];
export const LETTERS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ".split("");
/**
* Map a letter to a number in 0..25.
*
* @param {char} c
* @param {boolean} permissive - Case insensitive; don't throw errors on other chars.
* @returns {number}
*/
export function a2i(c, permissive=false) {
const i = Utils.ord(c);
if (i >= 65 && i <= 90) {
return i - 65;
}
if (permissive) {
// Allow case insensitivity
if (i >= 97 && i <= 122) {
return i - 97;
}
return -1;
}
throw new OperationError("a2i called on non-uppercase ASCII character");
}
/**
* Map a number in 0..25 to a letter.
*
* @param {number} i
* @returns {char}
*/
export function i2a(i) {
if (i >= 0 && i < 26) {
return Utils.chr(i+65);
}
throw new OperationError("i2a called on value outside 0..25");
}
/**
* A rotor in the Enigma machine.
*/
export class Rotor {
/**
* Rotor constructor.
*
* @param {string} wiring - A 26 character string of the wiring order.
* @param {string} steps - A 0..26 character string of stepping points.
* @param {char} ringSetting - The ring setting.
* @param {char} initialPosition - The initial position of the rotor.
*/
constructor(wiring, steps, ringSetting, initialPosition) {
if (!/^[A-Z]{26}$/.test(wiring)) {
throw new OperationError("Rotor wiring must be 26 unique uppercase letters");
}
if (!/^[A-Z]{0,26}$/.test(steps)) {
throw new OperationError("Rotor steps must be 0-26 unique uppercase letters");
}
if (!/^[A-Z]$/.test(ringSetting)) {
throw new OperationError("Rotor ring setting must be exactly one uppercase letter");
}
if (!/^[A-Z]$/.test(initialPosition)) {
throw new OperationError("Rotor initial position must be exactly one uppercase letter");
}
this.map = new Array(26);
this.revMap = new Array(26);
const uniq = {};
for (let i=0; i<LETTERS.length; i++) {
const a = a2i(LETTERS[i]);
const b = a2i(wiring[i]);
this.map[a] = b;
this.revMap[b] = a;
uniq[b] = true;
}
if (Object.keys(uniq).length !== LETTERS.length) {
throw new OperationError("Rotor wiring must have each letter exactly once");
}
const rs = a2i(ringSetting);
this.steps = new Set();
for (const x of steps) {
this.steps.add(Utils.mod(a2i(x) - rs, 26));
}
if (this.steps.size !== steps.length) {
// This isn't strictly fatal, but it's probably a mistake
throw new OperationError("Rotor steps must be unique");
}
this.pos = Utils.mod(a2i(initialPosition) - rs, 26);
}
/**
* Step the rotor forward by one.
*/
step() {
this.pos = Utils.mod(this.pos + 1, 26);
return this.pos;
}
/**
* Transform a character through this rotor forwards.
*
* @param {number} c - The character.
* @returns {number}
*/
transform(c) {
return Utils.mod(this.map[Utils.mod(c + this.pos, 26)] - this.pos, 26);
}
/**
* Transform a character through this rotor backwards.
*
* @param {number} c - The character.
* @returns {number}
*/
revTransform(c) {
return Utils.mod(this.revMap[Utils.mod(c + this.pos, 26)] - this.pos, 26);
}
}
/**
* Base class for plugboard and reflector (since these do effectively the same
* thing).
*/
class PairMapBase {
/**
* PairMapBase constructor.
*
* @param {string} pairs - A whitespace separated string of letter pairs to swap.
* @param {string} [name='PairMapBase'] - For errors, the name of this object.
*/
constructor(pairs, name="PairMapBase") {
// I've chosen to make whitespace significant here to make a) code and
// b) inputs easier to read
this.pairs = pairs;
this.map = {};
if (pairs === "") {
return;
}
pairs.split(/\s+/).forEach(pair => {
if (!/^[A-Z]{2}$/.test(pair)) {
throw new OperationError(name + " must be a whitespace-separated list of uppercase letter pairs");
}
const a = a2i(pair[0]), b = a2i(pair[1]);
if (a === b) {
// self-stecker
return;
}
if (Object.prototype.hasOwnProperty.call(this.map, a)) {
throw new OperationError(`${name} connects ${pair[0]} more than once`);
}
if (Object.prototype.hasOwnProperty.call(this.map, b)) {
throw new OperationError(`${name} connects ${pair[1]} more than once`);
}
this.map[a] = b;
this.map[b] = a;
});
}
/**
* Transform a character through this object.
* Returns other characters unchanged.
*
* @param {number} c - The character.
* @returns {number}
*/
transform(c) {
if (!Object.prototype.hasOwnProperty.call(this.map, c)) {
return c;
}
return this.map[c];
}
/**
* Alias for transform, to allow interchangeable use with rotors.
*
* @param {number} c - The character.
* @returns {number}
*/
revTransform(c) {
return this.transform(c);
}
}
/**
* Reflector. PairMapBase but requires that all characters are accounted for.
*
* Includes a couple of optimisations on that basis.
*/
export class Reflector extends PairMapBase {
/**
* Reflector constructor. See PairMapBase.
* Additional restriction: every character must be accounted for.
*/
constructor(pairs) {
super(pairs, "Reflector");
const s = Object.keys(this.map).length;
if (s !== 26) {
throw new OperationError("Reflector must have exactly 13 pairs covering every letter");
}
const optMap = new Array(26);
for (const x of Object.keys(this.map)) {
optMap[x] = this.map[x];
}
this.map = optMap;
}
/**
* Transform a character through this object.
*
* @param {number} c - The character.
* @returns {number}
*/
transform(c) {
return this.map[c];
}
}
/**
* Plugboard. Unmodified PairMapBase.
*/
export class Plugboard extends PairMapBase {
/**
* Plugboard constructor. See PairMapbase.
*/
constructor(pairs) {
super(pairs, "Plugboard");
}
}
/**
* Base class for the Enigma machine itself. Holds rotors, a reflector, and a plugboard.
*/
export class EnigmaBase {
/**
* EnigmaBase constructor.
*
* @param {Object[]} rotors - List of Rotors.
* @param {Object} reflector - A Reflector.
* @param {Plugboard} plugboard - A Plugboard.
*/
constructor(rotors, reflector, plugboard) {
this.rotors = rotors;
this.rotorsRev = [].concat(rotors).reverse();
this.reflector = reflector;
this.plugboard = plugboard;
}
/**
* Step the rotors forward by one.
*
* This happens before the output character is generated.
*
* Note that rotor 4, if it's there, never steps.
*
* Why is all the logic in EnigmaBase and not a nice neat method on
* Rotor that knows when it should advance the next item?
* Because the double stepping anomaly is a thing. tl;dr if the left rotor
* should step the next time the middle rotor steps, the middle rotor will
* immediately step.
*/
step() {
const r0 = this.rotors[0];
const r1 = this.rotors[1];
r0.step();
// The second test here is the double-stepping anomaly
if (r0.steps.has(r0.pos) || r1.steps.has(Utils.mod(r1.pos + 1, 26))) {
r1.step();
if (r1.steps.has(r1.pos)) {
const r2 = this.rotors[2];
r2.step();
}
}
}
/**
* Encrypt (or decrypt) some data.
* Takes an arbitrary string and runs the Engima machine on that data from
* *its current state*, and outputs the result. Non-alphabetic characters
* are returned unchanged.
*
* @param {string} input - Data to encrypt.
* @returns {string}
*/
crypt(input) {
let result = "";
for (const c of input) {
let letter = a2i(c, true);
if (letter === -1) {
result += c;
continue;
}
// First, step the rotors forward.
this.step();
// Now, run through the plugboard.
letter = this.plugboard.transform(letter);
// Then through each wheel in sequence, through the reflector, and
// backwards through the wheels again.
for (const rotor of this.rotors) {
letter = rotor.transform(letter);
}
letter = this.reflector.transform(letter);
for (const rotor of this.rotorsRev) {
letter = rotor.revTransform(letter);
}
// Finally, back through the plugboard.
letter = this.plugboard.revTransform(letter);
result += i2a(letter);
}
return result;
}
}
/**
* The Enigma machine itself. Holds 3-4 rotors, a reflector, and a plugboard.
*/
export class EnigmaMachine extends EnigmaBase {
/**
* EnigmaMachine constructor.
*
* @param {Object[]} rotors - List of Rotors.
* @param {Object} reflector - A Reflector.
* @param {Plugboard} plugboard - A Plugboard.
*/
constructor(rotors, reflector, plugboard) {
super(rotors, reflector, plugboard);
if (rotors.length !== 3 && rotors.length !== 4) {
throw new OperationError("Enigma must have 3 or 4 rotors");
}
}
}

18
src/core/lib/Extract.mjs
View file

@ -39,3 +39,21 @@ export function search (input, searchRegex, removeRegex, includeTotal) {
return output;
}
/**
* URL regular expression
*/
const protocol = "[A-Z]+://",
hostname = "[-\\w]+(?:\\.\\w[-\\w]*)+",
port = ":\\d+",
path = "/[^.!,?\"<>\\[\\]{}\\s\\x7F-\\xFF]*" +
"(?:[.!,?]+[^.!,?\"<>\\[\\]{}\\s\\x7F-\\xFF]+)*";
export const URL_REGEX = new RegExp(protocol + hostname + "(?:" + port + ")?(?:" + path + ")?", "ig");
/**
* Domain name regular expression
*/
export const DOMAIN_REGEX = /\b((?=[a-z0-9-]{1,63}\.)(xn--)?[a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,63}\b/ig;

2085
src/core/lib/FileSignatures.mjs Normal file
View file

File diff suppressed because it is too large Load diff

267
src/core/lib/FileType.mjs Normal file
View file

@ -0,0 +1,267 @@
/**
* File type functions
*
* @author n1474335 [n1474335@gmail.com]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*
*/
import {FILE_SIGNATURES} from "./FileSignatures.mjs";
import {sendStatusMessage} from "../Utils.mjs";
/**
* Checks whether a signature matches a buffer.
*
* @param {Object|Object[]} sig - A dictionary of offsets with values assigned to them.
* These values can be numbers for static checks, arrays of potential valid matches,
* or bespoke functions to check the validity of the buffer value at that offset.
* @param {Uint8Array} buf
* @param {number} [offset=0] Where in the buffer to start searching from
* @returns {boolean}
*/
function signatureMatches(sig, buf, offset=0) {
// Using a length check seems to be more performant than `sig instanceof Array`
if (sig.length) {
// sig is an Array - return true if any of them match
// The following `reduce` method is nice, but performance matters here, so we
// opt for a faster, if less elegant, for loop.
// return sig.reduce((acc, s) => acc || bytesMatch(s, buf, offset), false);
for (let i = 0; i < sig.length; i++) {
if (bytesMatch(sig[i], buf, offset)) return true;
}
return false;
} else {
return bytesMatch(sig, buf, offset);
}
}
/**
* Checks whether a set of bytes match the given buffer.
*
* @param {Object} sig - A dictionary of offsets with values assigned to them.
* These values can be numbers for static checks, arrays of potential valid matches,
* or bespoke functions to check the validity of the buffer value at that offset.
* @param {Uint8Array} buf
* @param {number} [offset=0] Where in the buffer to start searching from
* @returns {boolean}
*/
function bytesMatch(sig, buf, offset=0) {
for (const sigoffset in sig) {
const pos = parseInt(sigoffset, 10) + offset;
switch (typeof sig[sigoffset]) {
case "number": // Static check
if (buf[pos] !== sig[sigoffset])
return false;
break;
case "object": // Array of options
if (sig[sigoffset].indexOf(buf[pos]) < 0)
return false;
break;
case "function": // More complex calculation
if (!sig[sigoffset](buf[pos]))
return false;
break;
default:
throw new Error(`Unrecognised signature type at offset ${sigoffset}`);
}
}
return true;
}
/**
* Given a buffer, detects magic byte sequences at specific positions and returns the
* extension and mime type.
*
* @param {Uint8Array|ArrayBuffer} buf
* @param {string[]} [categories=All] - Which categories of file to look for
* @returns {Object[]} types
* @returns {string} type.name - Name of file type
* @returns {string} type.ext - File extension
* @returns {string} type.mime - Mime type
* @returns {string} [type.desc] - Description
*/
export function detectFileType(buf, categories=Object.keys(FILE_SIGNATURES)) {
if (buf instanceof ArrayBuffer) {
buf = new Uint8Array(buf);
}
if (!(buf && buf.length > 1)) {
return [];
}
const matchingFiles = [];
const signatures = {};
for (const cat in FILE_SIGNATURES) {
if (categories.includes(cat)) {
signatures[cat] = FILE_SIGNATURES[cat];
}
}
for (const cat in signatures) {
const category = signatures[cat];
category.forEach(filetype => {
if (signatureMatches(filetype.signature, buf)) {
matchingFiles.push(filetype);
}
});
}
return matchingFiles;
}
/**
* Given a buffer, searches for magic byte sequences at all possible positions and returns
* the extensions and mime types.
*
* @param {Uint8Array} buf
* @param {string[]} [categories=All] - Which categories of file to look for
* @returns {Object[]} foundFiles
* @returns {number} foundFiles.offset - The position in the buffer at which this file was found
* @returns {Object} foundFiles.fileDetails
* @returns {string} foundFiles.fileDetails.name - Name of file type
* @returns {string} foundFiles.fileDetails.ext - File extension
* @returns {string} foundFiles.fileDetails.mime - Mime type
* @returns {string} [foundFiles.fileDetails.desc] - Description
*/
export function scanForFileTypes(buf, categories=Object.keys(FILE_SIGNATURES)) {
if (!(buf && buf.length > 1)) {
return [];
}
const foundFiles = [];
const signatures = {};
for (const cat in FILE_SIGNATURES) {
if (categories.includes(cat)) {
signatures[cat] = FILE_SIGNATURES[cat];
}
}
for (const cat in signatures) {
const category = signatures[cat];
for (let i = 0; i < category.length; i++) {
const filetype = category[i];
const sigs = filetype.signature.length ? filetype.signature : [filetype.signature];
sigs.forEach(sig => {
let pos = 0;
while ((pos = locatePotentialSig(buf, sig, pos)) >= 0) {
if (bytesMatch(sig, buf, pos)) {
sendStatusMessage(`Found potential signature for ${filetype.name} at pos ${pos}`);
foundFiles.push({
offset: pos,
fileDetails: filetype
});
}
pos++;
}
});
}
}
// Return found files in order of increasing offset
return foundFiles.sort((a, b) => {
return a.offset - b.offset;
});
}
/**
* Fastcheck function to quickly scan the buffer for the first byte in a signature.
*
* @param {Uint8Array} buf - The buffer to search
* @param {Object} sig - A single signature object (Not an array of signatures)
* @param {number} offset - Where to start search from
* @returs {number} The position of the match or -1 if one cannot be found.
*/
function locatePotentialSig(buf, sig, offset) {
// Find values for first key and value in sig
const k = parseInt(Object.keys(sig)[0], 10);
const v = Object.values(sig)[0];
switch (typeof v) {
case "number":
return buf.indexOf(v, offset + k) - k;
case "object":
for (let i = offset + k; i < buf.length; i++) {
if (v.indexOf(buf[i]) >= 0) return i - k;
}
return -1;
case "function":
for (let i = offset + k; i < buf.length; i++) {
if (v(buf[i])) return i - k;
}
return -1;
default:
throw new Error("Unrecognised signature type");
}
}
/**
* Detects whether the given buffer is a file of the type specified.
*
* @param {string|RegExp} type
* @param {Uint8Array|ArrayBuffer} buf
* @returns {string|false} The mime type or false if the type does not match
*/
export function isType(type, buf) {
const types = detectFileType(buf);
if (!(types && types.length)) return false;
if (typeof type === "string") {
return types.reduce((acc, t) => {
const mime = t.mime.startsWith(type) ? t.mime : false;
return acc || mime;
}, false);
} else if (type instanceof RegExp) {
return types.reduce((acc, t) => {
const mime = type.test(t.mime) ? t.mime : false;
return acc || mime;
}, false);
} else {
throw new Error("Invalid type input.");
}
}
/**
* Detects whether the given buffer contains an image file.
*
* @param {Uint8Array|ArrayBuffer} buf
* @returns {string|false} The mime type or false if the type does not match
*/
export function isImage(buf) {
return isType("image", buf);
}
/**
* Attempts to extract a file from a data stream given its offset and extractor function.
*
* @param {Uint8Array} bytes
* @param {Object} fileDetail
* @param {string} fileDetail.mime
* @param {string} fileDetail.extension
* @param {Function} fileDetail.extractor
* @param {number} offset
* @returns {File}
*/
export function extractFile(bytes, fileDetail, offset) {
if (fileDetail.extractor) {
sendStatusMessage(`Attempting to extract ${fileDetail.name} at pos ${offset}...`);
const fileData = fileDetail.extractor(bytes, offset);
const ext = fileDetail.extension.split(",")[0];
return new File([fileData], `extracted_at_0x${offset.toString(16)}.${ext}`, {
type: fileDetail.mime
});
}
throw new Error(`No extraction algorithm available for "${fileDetail.mime}" files`);
}

4
src/core/lib/Hash.mjs
View file

@ -7,8 +7,8 @@
* @license Apache-2.0
*/
import Utils from "../Utils";
import CryptoApi from "crypto-api/src/crypto-api";
import Utils from "../Utils.mjs";
import CryptoApi from "crypto-api/src/crypto-api.mjs";
/**

10
src/core/lib/Hex.mjs
View file

@ -6,13 +6,13 @@
* @license Apache-2.0
*/
import Utils from "../Utils";
import Utils from "../Utils.mjs";
/**
* Convert a byte array into a hex string.
*
* @param {Uint8Array|byteArray} data
* @param {byteArray|Uint8Array|ArrayBuffer} data
* @param {string} [delim=" "]
* @param {number} [padding=2]
* @returns {string}
@ -26,6 +26,7 @@ import Utils from "../Utils";
*/
export function toHex(data, delim=" ", padding=2) {
if (!data) return "";
if (data instanceof ArrayBuffer) data = new Uint8Array(data);
let output = "";
@ -47,7 +48,7 @@ export function toHex(data, delim=" ", padding=2) {
/**
* Convert a byte array into a hex string as efficiently as possible with no options.
*
* @param {byteArray} data
* @param {byteArray|Uint8Array|ArrayBuffer} data
* @returns {string}
*
* @example
@ -56,6 +57,7 @@ export function toHex(data, delim=" ", padding=2) {
*/
export function toHexFast(data) {
if (!data) return "";
if (data instanceof ArrayBuffer) data = new Uint8Array(data);
const output = [];
@ -100,7 +102,7 @@ export function fromHex(data, delim="Auto", byteLen=2) {
/**
* To Hexadecimal delimiters.
*/
export const TO_HEX_DELIM_OPTIONS = ["Space", "Comma", "Semi-colon", "Colon", "Line feed", "CRLF", "0x", "\\x", "None"];
export const TO_HEX_DELIM_OPTIONS = ["Space", "Percent", "Comma", "Semi-colon", "Colon", "Line feed", "CRLF", "0x", "\\x", "None"];
/**

10
src/core/lib/IP.mjs
View file

@ -8,8 +8,8 @@
* @license Apache-2.0
*/
import Utils from "../Utils";
import OperationError from "../errors/OperationError";
import Utils from "../Utils.mjs";
import OperationError from "../errors/OperationError.mjs";
/**
* Parses an IPv4 CIDR range (e.g. 192.168.0.0/24) and displays information about it.
@ -241,7 +241,7 @@ export function ipv6ListedRange(match, includeNetworkInfo) {
ipv6List = ipv6List.filter(function(str) {
return str.trim();
});
for (let i =0; i < ipv6List.length; i++){
for (let i =0; i < ipv6List.length; i++) {
ipv6List[i] = ipv6List[i].trim();
}
const ipv6CidrList = ipv6List.filter(function(a) {
@ -502,8 +502,8 @@ export function ipv6Compare(a, b) {
const a_ = strToIpv6(a),
b_ = strToIpv6(b);
for (let i = 0; i < a_.length; i++){
if (a_[i] !== b_[i]){
for (let i = 0; i < a_.length; i++) {
if (a_[i] !== b_[i]) {
return a_[i] - b_[i];
}
}

251
src/core/lib/ImageManipulation.mjs Normal file
View file

@ -0,0 +1,251 @@
/**
* Image manipulation resources
*
* @author j433866 [j433866@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
/**
* Gaussian blurs an image.
*
* @param {jimp} input
* @param {number} radius
* @param {boolean} fast
* @returns {jimp}
*/
export function gaussianBlur (input, radius) {
try {
// From http://blog.ivank.net/fastest-gaussian-blur.html
const boxes = boxesForGauss(radius, 3);
for (let i = 0; i < 3; i++) {
input = boxBlur(input, (boxes[i] - 1) / 2);
}
} catch (err) {
throw new OperationError(`Error blurring image. (${err})`);
}
return input;
}
/**
*
* @param {number} radius
* @param {number} numBoxes
* @returns {Array}
*/
function boxesForGauss(radius, numBoxes) {
const idealWidth = Math.sqrt((12 * radius * radius / numBoxes) + 1);
let wl = Math.floor(idealWidth);
if (wl % 2 === 0) {
wl--;
}
const wu = wl + 2;
const mIdeal = (12 * radius * radius - numBoxes * wl * wl - 4 * numBoxes * wl - 3 * numBoxes) / (-4 * wl - 4);
const m = Math.round(mIdeal);
const sizes = [];
for (let i = 0; i < numBoxes; i++) {
sizes.push(i < m ? wl : wu);
}
return sizes;
}
/**
* Applies a box blur effect to the image
*
* @param {jimp} source
* @param {number} radius
* @returns {jimp}
*/
function boxBlur (source, radius) {
const width = source.bitmap.width;
const height = source.bitmap.height;
let output = source.clone();
output = boxBlurH(source, output, width, height, radius);
source = boxBlurV(output, source, width, height, radius);
return source;
}
/**
* Applies the horizontal blur
*
* @param {jimp} source
* @param {jimp} output
* @param {number} width
* @param {number} height
* @param {number} radius
* @returns {jimp}
*/
function boxBlurH (source, output, width, height, radius) {
const iarr = 1 / (radius + radius + 1);
for (let i = 0; i < height; i++) {
let ti = 0,
li = ti,
ri = ti + radius;
const idx = source.getPixelIndex(ti, i);
const firstValRed = source.bitmap.data[idx],
firstValGreen = source.bitmap.data[idx + 1],
firstValBlue = source.bitmap.data[idx + 2],
firstValAlpha = source.bitmap.data[idx + 3];
const lastIdx = source.getPixelIndex(width - 1, i),
lastValRed = source.bitmap.data[lastIdx],
lastValGreen = source.bitmap.data[lastIdx + 1],
lastValBlue = source.bitmap.data[lastIdx + 2],
lastValAlpha = source.bitmap.data[lastIdx + 3];
let red = (radius + 1) * firstValRed;
let green = (radius + 1) * firstValGreen;
let blue = (radius + 1) * firstValBlue;
let alpha = (radius + 1) * firstValAlpha;
for (let j = 0; j < radius; j++) {
const jIdx = source.getPixelIndex(ti + j, i);
red += source.bitmap.data[jIdx];
green += source.bitmap.data[jIdx + 1];
blue += source.bitmap.data[jIdx + 2];
alpha += source.bitmap.data[jIdx + 3];
}
for (let j = 0; j <= radius; j++) {
const jIdx = source.getPixelIndex(ri++, i);
red += source.bitmap.data[jIdx] - firstValRed;
green += source.bitmap.data[jIdx + 1] - firstValGreen;
blue += source.bitmap.data[jIdx + 2] - firstValBlue;
alpha += source.bitmap.data[jIdx + 3] - firstValAlpha;
const tiIdx = source.getPixelIndex(ti++, i);
output.bitmap.data[tiIdx] = Math.round(red * iarr);
output.bitmap.data[tiIdx + 1] = Math.round(green * iarr);
output.bitmap.data[tiIdx + 2] = Math.round(blue * iarr);
output.bitmap.data[tiIdx + 3] = Math.round(alpha * iarr);
}
for (let j = radius + 1; j < width - radius; j++) {
const riIdx = source.getPixelIndex(ri++, i);
const liIdx = source.getPixelIndex(li++, i);
red += source.bitmap.data[riIdx] - source.bitmap.data[liIdx];
green += source.bitmap.data[riIdx + 1] - source.bitmap.data[liIdx + 1];
blue += source.bitmap.data[riIdx + 2] - source.bitmap.data[liIdx + 2];
alpha += source.bitmap.data[riIdx + 3] - source.bitmap.data[liIdx + 3];
const tiIdx = source.getPixelIndex(ti++, i);
output.bitmap.data[tiIdx] = Math.round(red * iarr);
output.bitmap.data[tiIdx + 1] = Math.round(green * iarr);
output.bitmap.data[tiIdx + 2] = Math.round(blue * iarr);
output.bitmap.data[tiIdx + 3] = Math.round(alpha * iarr);
}
for (let j = width - radius; j < width; j++) {
const liIdx = source.getPixelIndex(li++, i);
red += lastValRed - source.bitmap.data[liIdx];
green += lastValGreen - source.bitmap.data[liIdx + 1];
blue += lastValBlue - source.bitmap.data[liIdx + 2];
alpha += lastValAlpha - source.bitmap.data[liIdx + 3];
const tiIdx = source.getPixelIndex(ti++, i);
output.bitmap.data[tiIdx] = Math.round(red * iarr);
output.bitmap.data[tiIdx + 1] = Math.round(green * iarr);
output.bitmap.data[tiIdx + 2] = Math.round(blue * iarr);
output.bitmap.data[tiIdx + 3] = Math.round(alpha * iarr);
}
}
return output;
}
/**
* Applies the vertical blur
*
* @param {jimp} source
* @param {jimp} output
* @param {number} width
* @param {number} height
* @param {number} radius
* @returns {jimp}
*/
function boxBlurV (source, output, width, height, radius) {
const iarr = 1 / (radius + radius + 1);
for (let i = 0; i < width; i++) {
let ti = 0,
li = ti,
ri = ti + radius;
const idx = source.getPixelIndex(i, ti);
const firstValRed = source.bitmap.data[idx],
firstValGreen = source.bitmap.data[idx + 1],
firstValBlue = source.bitmap.data[idx + 2],
firstValAlpha = source.bitmap.data[idx + 3];
const lastIdx = source.getPixelIndex(i, height - 1),
lastValRed = source.bitmap.data[lastIdx],
lastValGreen = source.bitmap.data[lastIdx + 1],
lastValBlue = source.bitmap.data[lastIdx + 2],
lastValAlpha = source.bitmap.data[lastIdx + 3];
let red = (radius + 1) * firstValRed;
let green = (radius + 1) * firstValGreen;
let blue = (radius + 1) * firstValBlue;
let alpha = (radius + 1) * firstValAlpha;
for (let j = 0; j < radius; j++) {
const jIdx = source.getPixelIndex(i, ti + j);
red += source.bitmap.data[jIdx];
green += source.bitmap.data[jIdx + 1];
blue += source.bitmap.data[jIdx + 2];
alpha += source.bitmap.data[jIdx + 3];
}
for (let j = 0; j <= radius; j++) {
const riIdx = source.getPixelIndex(i, ri++);
red += source.bitmap.data[riIdx] - firstValRed;
green += source.bitmap.data[riIdx + 1] - firstValGreen;
blue += source.bitmap.data[riIdx + 2] - firstValBlue;
alpha += source.bitmap.data[riIdx + 3] - firstValAlpha;
const tiIdx = source.getPixelIndex(i, ti++);
output.bitmap.data[tiIdx] = Math.round(red * iarr);
output.bitmap.data[tiIdx + 1] = Math.round(green * iarr);
output.bitmap.data[tiIdx + 2] = Math.round(blue * iarr);
output.bitmap.data[tiIdx + 3] = Math.round(alpha * iarr);
}
for (let j = radius + 1; j < height - radius; j++) {
const riIdx = source.getPixelIndex(i, ri++);
const liIdx = source.getPixelIndex(i, li++);
red += source.bitmap.data[riIdx] - source.bitmap.data[liIdx];
green += source.bitmap.data[riIdx + 1] - source.bitmap.data[liIdx + 1];
blue += source.bitmap.data[riIdx + 2] - source.bitmap.data[liIdx + 2];
alpha += source.bitmap.data[riIdx + 3] - source.bitmap.data[liIdx + 3];
const tiIdx = source.getPixelIndex(i, ti++);
output.bitmap.data[tiIdx] = Math.round(red * iarr);
output.bitmap.data[tiIdx + 1] = Math.round(green * iarr);
output.bitmap.data[tiIdx + 2] = Math.round(blue * iarr);
output.bitmap.data[tiIdx + 3] = Math.round(alpha * iarr);
}
for (let j = height - radius; j < height; j++) {
const liIdx = source.getPixelIndex(i, li++);
red += lastValRed - source.bitmap.data[liIdx];
green += lastValGreen - source.bitmap.data[liIdx + 1];
blue += lastValBlue - source.bitmap.data[liIdx + 2];
alpha += lastValAlpha - source.bitmap.data[liIdx + 3];
const tiIdx = source.getPixelIndex(i, ti++);
output.bitmap.data[tiIdx] = Math.round(red * iarr);
output.bitmap.data[tiIdx + 1] = Math.round(green * iarr);
output.bitmap.data[tiIdx + 2] = Math.round(blue * iarr);
output.bitmap.data[tiIdx + 3] = Math.round(alpha * iarr);
}
}
return output;
}

230
src/core/lib/LoremIpsum.mjs Normal file
View file

@ -0,0 +1,230 @@
/**
* Lorem Ipsum generator.
*
* @author Klaxon [klaxon@veyr.com]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
/**
* Generate lorem ipsum paragraphs.
*
* @param {number} length
* @returns {string}
*/
export function GenerateParagraphs(length=3) {
const paragraphs = [];
while (paragraphs.length < length) {
const paragraphLength = getRandomLength(PARAGRAPH_LENGTH_MEAN, PARAGRAPH_LENGTH_STD_DEV);
const sentences = [];
while (sentences.length < paragraphLength) {
const sentenceLength = getRandomLength(SENTENCE_LENGTH_MEAN, SENTENCE_LENGTH_STD_DEV);
const sentence = getWords(sentenceLength);
sentences.push(formatSentence(sentence));
}
paragraphs.push(formatParagraph(sentences));
}
paragraphs[paragraphs.length-1] = paragraphs[paragraphs.length-1].slice(0, -2);
paragraphs[0] = replaceStart(paragraphs[0]);
return paragraphs.join("");
}
/**
* Generate lorem ipsum sentences.
*
* @param {number} length
* @returns {string}
*/
export function GenerateSentences(length=3) {
const sentences = [];
while (sentences.length < length) {
const sentenceLength = getRandomLength(SENTENCE_LENGTH_MEAN, SENTENCE_LENGTH_STD_DEV);
const sentence = getWords(sentenceLength);
sentences.push(formatSentence(sentence));
}
const paragraphs = sentencesToParagraphs(sentences);
return paragraphs.join("");
}
/**
* Generate lorem ipsum words.
*
* @param {number} length
* @returns {string}
*/
export function GenerateWords(length=3) {
const words = getWords(length);
const sentences = wordsToSentences(words);
const paragraphs = sentencesToParagraphs(sentences);
return paragraphs.join("");
}
/**
* Generate lorem ipsum bytes.
*
* @param {number} length
* @returns {string}
*/
export function GenerateBytes(length=3) {
const str = GenerateWords(length/3);
return str.slice(0, length);
}
/**
* Get array of randomly selected words from the lorem ipsum wordList.
*
* @param {number} length
* @returns {string[]}
* @private
*/
function getWords(length=3) {
const words = [];
let word;
let previousWord;
while (words.length < length) {
do {
word = wordList[Math.floor(Math.random() * wordList.length)];
} while (previousWord === word);
words.push(word);
previousWord = word;
}
return words;
}
/**
* Convert an array of words into an array of sentences
*
* @param {string[]} words
* @returns {string[]}
* @private
*/
function wordsToSentences(words) {
const sentences = [];
while (words.length > 0) {
const sentenceLength = getRandomLength(SENTENCE_LENGTH_MEAN, SENTENCE_LENGTH_STD_DEV);
if (sentenceLength <= words.length) {
sentences.push(formatSentence(words.splice(0, sentenceLength)));
} else {
sentences.push(formatSentence(words.splice(0, words.length)));
}
}
return sentences;
}
/**
* Convert an array of sentences into an array of paragraphs
*
* @param {string[]} sentences
* @returns {string[]}
* @private
*/
function sentencesToParagraphs(sentences) {
const paragraphs = [];
while (sentences.length > 0) {
const paragraphLength = getRandomLength(PARAGRAPH_LENGTH_MEAN, PARAGRAPH_LENGTH_STD_DEV);
paragraphs.push(formatParagraph(sentences.splice(0, paragraphLength)));
}
paragraphs[paragraphs.length-1] = paragraphs[paragraphs.length-1].slice(0, -1);
paragraphs[0] = replaceStart(paragraphs[0]);
return paragraphs;
}
/**
* Format an array of words into a sentence.
*
* @param {string[]} words
* @returns {string}
* @private
*/
function formatSentence(words) {
// 0.35 chance of a comma being added randomly to the sentence.
if (Math.random() < PROBABILITY_OF_A_COMMA) {
const pos = Math.round(Math.random()*(words.length-1));
words[pos] +=",";
}
let sentence = words.join(" ");
sentence = sentence.charAt(0).toUpperCase() + sentence.slice(1);
sentence += ".";
return sentence;
}
/**
* Format an array of sentences into a paragraph.
*
* @param {string[]} sentences
* @returns {string}
* @private
*/
function formatParagraph(sentences) {
let paragraph = sentences.join(" ");
paragraph += "\n\n";
return paragraph;
}
/**
* Get a random number based on a mean and standard deviation.
*
* @param {number} mean
* @param {number} stdDev
* @returns {number}
* @private
*/
function getRandomLength(mean, stdDev) {
let length;
do {
length = Math.round((Math.random()*2-1)+(Math.random()*2-1)+(Math.random()*2-1)*stdDev+mean);
} while (length <= 0);
return length;
}
/**
* Replace first 5 words with "Lorem ipsum dolor sit amet"
*
* @param {string[]} str
* @returns {string[]}
* @private
*/
function replaceStart(str) {
let words = str.split(" ");
if (words.length > 5) {
words.splice(0, 5, "Lorem", "ipsum", "dolor", "sit", "amet");
return words.join(" ");
} else {
const lorem = ["Lorem", "ipsum", "dolor", "sit", "amet"];
words = lorem.slice(0, words.length);
str = words.join(" ");
str += ".";
return str;
}
}
const SENTENCE_LENGTH_MEAN = 15;
const SENTENCE_LENGTH_STD_DEV = 9;
const PARAGRAPH_LENGTH_MEAN = 5;
const PARAGRAPH_LENGTH_STD_DEV = 2;
const PROBABILITY_OF_A_COMMA = 0.35;
const wordList = [
"ad", "adipisicing", "aliqua", "aliquip", "amet", "anim",
"aute", "cillum", "commodo", "consectetur", "consequat", "culpa",
"cupidatat", "deserunt", "do", "dolor", "dolore", "duis",
"ea", "eiusmod", "elit", "enim", "esse", "est",
"et", "eu", "ex", "excepteur", "exercitation", "fugiat",
"id", "in", "incididunt", "ipsum", "irure", "labore",
"laboris", "laborum", "Lorem", "magna", "minim", "mollit",
"nisi", "non", "nostrud", "nulla", "occaecat", "officia",
"pariatur", "proident", "qui", "quis", "reprehenderit", "sint",
"sit", "sunt", "tempor", "ullamco", "ut", "velit",
"veniam", "voluptate",
];

492
src/core/lib/Magic.mjs
View file

@ -1,7 +1,8 @@
import OperationConfig from "../config/OperationConfig.json";
import Utils from "../Utils";
import Recipe from "../Recipe";
import Dish from "../Dish";
import Utils, { isWorkerEnvironment } from "../Utils.mjs";
import Recipe from "../Recipe.mjs";
import Dish from "../Dish.mjs";
import {detectFileType} from "./FileType.mjs";
import chiSquared from "chi-squared";
/**
@ -92,7 +93,14 @@ class Magic {
* @returns {string} [type.desc] - Description
*/
detectFileType() {
return Magic.magicFileType(this.inputBuffer);
const fileType = detectFileType(this.inputBuffer);
if (!fileType.length) return null;
return {
ext: fileType[0].extension,
mime: fileType[0].mime,
desc: fileType[0].description
};
}
/**
@ -265,9 +273,10 @@ class Magic {
* performance)
* @param {Object[]} [recipeConfig=[]] - The recipe configuration up to this point
* @param {boolean} [useful=false] - Whether the current recipe should be scored highly
* @param {string} [crib=null] - The regex crib provided by the user, for filtering the operation output
* @returns {Object[]} - A sorted list of the recipes most likely to result in correct decoding
*/
async speculativeExecution(depth=0, extLang=false, intensive=false, recipeConfig=[], useful=false) {
async speculativeExecution(depth=0, extLang=false, intensive=false, recipeConfig=[], useful=false, crib=null) {
if (depth < 0) return [];
// Find any operations that can be run on this data
@ -284,9 +293,9 @@ class Magic {
isUTF8: this.isUTF8(),
entropy: this.calcEntropy(),
matchingOps: matchingOps,
useful: useful
useful: useful,
matchesCrib: crib && crib.test(this.inputStr)
});
const prevOp = recipeConfig[recipeConfig.length - 1];
// Execute each of the matching operations, then recursively call the speculativeExecution()
@ -303,9 +312,14 @@ class Magic {
return;
}
// If the recipe returned an empty buffer, do not continue
if (_buffersEqual(output, new ArrayBuffer())) {
return;
}
const magic = new Magic(output, this.opPatterns),
speculativeResults = await magic.speculativeExecution(
depth-1, extLang, intensive, [...recipeConfig, opConfig], op.useful);
depth-1, extLang, intensive, [...recipeConfig, opConfig], op.useful, crib);
results = results.concat(speculativeResults);
}));
@ -317,14 +331,14 @@ class Magic {
await Promise.all(bfEncodings.map(async enc => {
const magic = new Magic(enc.data, this.opPatterns),
bfResults = await magic.speculativeExecution(
depth-1, extLang, false, [...recipeConfig, enc.conf]);
depth-1, extLang, false, [...recipeConfig, enc.conf], false, crib);
results = results.concat(bfResults);
}));
}
// Prune branches that result in unhelpful outputs
results = results.filter(r =>
const prunedResults = results.filter(r =>
(r.useful || r.data.length > 0) && // The operation resulted in ""
( // One of the following must be true
r.languageScores[0].probability > 0 || // Some kind of language was found
@ -335,7 +349,7 @@ class Magic {
);
// Return a sorted list of possible recipes along with their properties
return results.sort((a, b) => {
return prunedResults.sort((a, b) => {
// Each option is sorted based on its most likely language (lower is better)
let aScore = a.languageScores[0].score,
bScore = b.languageScores[0].score;
@ -381,12 +395,17 @@ class Magic {
const dish = new Dish();
dish.set(input, Dish.ARRAY_BUFFER);
if (ENVIRONMENT_IS_WORKER()) self.loadRequiredModules(recipeConfig);
if (isWorkerEnvironment()) self.loadRequiredModules(recipeConfig);
const recipe = new Recipe(recipeConfig);
try {
await recipe.execute(dish);
return dish.get(Dish.ARRAY_BUFFER);
// Return an empty buffer if the recipe did not run to completion
if (recipe.lastRunOp === recipe.opList[recipe.opList.length - 1]) {
return dish.get(Dish.ARRAY_BUFFER);
} else {
return new ArrayBuffer();
}
} catch (err) {
// If there are errors, return an empty buffer
return new ArrayBuffer();
@ -431,7 +450,7 @@ class Magic {
const opPatterns = [];
for (const op in OperationConfig) {
if (!OperationConfig[op].hasOwnProperty("patterns")) continue;
if (!("patterns" in OperationConfig[op])) continue;
OperationConfig[op].patterns.forEach(pattern => {
opPatterns.push({
@ -784,452 +803,9 @@ class Magic {
}[code];
}
/**
* Given a buffer, detects magic byte sequences at specific positions and returns the
* extension and mime type.
*
* @param {Uint8Array} buf
* @returns {Object} type
* @returns {string} type.ext - File extension
* @returns {string} type.mime - Mime type
* @returns {string} [type.desc] - Description
*/
static magicFileType(buf) {
if (!(buf && buf.length > 1)) {
return null;
}
if (buf[0] === 0xFF && buf[1] === 0xD8 && buf[2] === 0xFF) {
return {
ext: "jpg",
mime: "image/jpeg"
};
}
if (buf[0] === 0x89 && buf[1] === 0x50 && buf[2] === 0x4E && buf[3] === 0x47) {
return {
ext: "png",
mime: "image/png"
};
}
if (buf[0] === 0x47 && buf[1] === 0x49 && buf[2] === 0x46) {
return {
ext: "gif",
mime: "image/gif"
};
}
if (buf[8] === 0x57 && buf[9] === 0x45 && buf[10] === 0x42 && buf[11] === 0x50) {
return {
ext: "webp",
mime: "image/webp"
};
}
// needs to be before `tif` check
if (((buf[0] === 0x49 && buf[1] === 0x49 && buf[2] === 0x2A && buf[3] === 0x0) || (buf[0] === 0x4D && buf[1] === 0x4D && buf[2] === 0x0 && buf[3] === 0x2A)) && buf[8] === 0x43 && buf[9] === 0x52) {
return {
ext: "cr2",
mime: "image/x-canon-cr2"
};
}
if ((buf[0] === 0x49 && buf[1] === 0x49 && buf[2] === 0x2A && buf[3] === 0x0) || (buf[0] === 0x4D && buf[1] === 0x4D && buf[2] === 0x0 && buf[3] === 0x2A)) {
return {
ext: "tif",
mime: "image/tiff"
};
}
if (buf[0] === 0x42 && buf[1] === 0x4D) {
return {
ext: "bmp",
mime: "image/bmp"
};
}
if (buf[0] === 0x49 && buf[1] === 0x49 && buf[2] === 0xBC) {
return {
ext: "jxr",
mime: "image/vnd.ms-photo"
};
}
if (buf[0] === 0x38 && buf[1] === 0x42 && buf[2] === 0x50 && buf[3] === 0x53) {
return {
ext: "psd",
mime: "image/vnd.adobe.photoshop"
};
}
// needs to be before `zip` check
if (buf[0] === 0x50 && buf[1] === 0x4B && buf[2] === 0x3 && buf[3] === 0x4 && buf[30] === 0x6D && buf[31] === 0x69 && buf[32] === 0x6D && buf[33] === 0x65 && buf[34] === 0x74 && buf[35] === 0x79 && buf[36] === 0x70 && buf[37] === 0x65 && buf[38] === 0x61 && buf[39] === 0x70 && buf[40] === 0x70 && buf[41] === 0x6C && buf[42] === 0x69 && buf[43] === 0x63 && buf[44] === 0x61 && buf[45] === 0x74 && buf[46] === 0x69 && buf[47] === 0x6F && buf[48] === 0x6E && buf[49] === 0x2F && buf[50] === 0x65 && buf[51] === 0x70 && buf[52] === 0x75 && buf[53] === 0x62 && buf[54] === 0x2B && buf[55] === 0x7A && buf[56] === 0x69 && buf[57] === 0x70) {
return {
ext: "epub",
mime: "application/epub+zip"
};
}
if (buf[0] === 0x50 && buf[1] === 0x4B && (buf[2] === 0x3 || buf[2] === 0x5 || buf[2] === 0x7) && (buf[3] === 0x4 || buf[3] === 0x6 || buf[3] === 0x8)) {
return {
ext: "zip",
mime: "application/zip"
};
}
if (buf[257] === 0x75 && buf[258] === 0x73 && buf[259] === 0x74 && buf[260] === 0x61 && buf[261] === 0x72) {
return {
ext: "tar",
mime: "application/x-tar"
};
}
if (buf[0] === 0x52 && buf[1] === 0x61 && buf[2] === 0x72 && buf[3] === 0x21 && buf[4] === 0x1A && buf[5] === 0x7 && (buf[6] === 0x0 || buf[6] === 0x1)) {
return {
ext: "rar",
mime: "application/x-rar-compressed"
};
}
if (buf[0] === 0x1F && buf[1] === 0x8B && buf[2] === 0x8) {
return {
ext: "gz",
mime: "application/gzip"
};
}
if (buf[0] === 0x42 && buf[1] === 0x5A && buf[2] === 0x68) {
return {
ext: "bz2",
mime: "application/x-bzip2"
};
}
if (buf[0] === 0x37 && buf[1] === 0x7A && buf[2] === 0xBC && buf[3] === 0xAF && buf[4] === 0x27 && buf[5] === 0x1C) {
return {
ext: "7z",
mime: "application/x-7z-compressed"
};
}
if (buf[0] === 0x78 && buf[1] === 0x01) {
return {
ext: "dmg, zlib",
mime: "application/x-apple-diskimage, application/x-deflate"
};
}
if ((buf[0] === 0x0 && buf[1] === 0x0 && buf[2] === 0x0 && (buf[3] === 0x18 || buf[3] === 0x20) && buf[4] === 0x66 && buf[5] === 0x74 && buf[6] === 0x79 && buf[7] === 0x70) || (buf[0] === 0x33 && buf[1] === 0x67 && buf[2] === 0x70 && buf[3] === 0x35) || (buf[0] === 0x0 && buf[1] === 0x0 && buf[2] === 0x0 && buf[3] === 0x1C && buf[4] === 0x66 && buf[5] === 0x74 && buf[6] === 0x79 && buf[7] === 0x70 && buf[8] === 0x6D && buf[9] === 0x70 && buf[10] === 0x34 && buf[11] === 0x32 && buf[16] === 0x6D && buf[17] === 0x70 && buf[18] === 0x34 && buf[19] === 0x31 && buf[20] === 0x6D && buf[21] === 0x70 && buf[22] === 0x34 && buf[23] === 0x32 && buf[24] === 0x69 && buf[25] === 0x73 && buf[26] === 0x6F && buf[27] === 0x6D)) {
return {
ext: "mp4",
mime: "video/mp4"
};
}
if ((buf[0] === 0x0 && buf[1] === 0x0 && buf[2] === 0x0 && buf[3] === 0x1C && buf[4] === 0x66 && buf[5] === 0x74 && buf[6] === 0x79 && buf[7] === 0x70 && buf[8] === 0x4D && buf[9] === 0x34 && buf[10] === 0x56)) {
return {
ext: "m4v",
mime: "video/x-m4v"
};
}
if (buf[0] === 0x4D && buf[1] === 0x54 && buf[2] === 0x68 && buf[3] === 0x64) {
return {
ext: "mid",
mime: "audio/midi"
};
}
// needs to be before the `webm` check
if (buf[31] === 0x6D && buf[32] === 0x61 && buf[33] === 0x74 && buf[34] === 0x72 && buf[35] === 0x6f && buf[36] === 0x73 && buf[37] === 0x6B && buf[38] === 0x61) {
return {
ext: "mkv",
mime: "video/x-matroska"
};
}
if (buf[0] === 0x1A && buf[1] === 0x45 && buf[2] === 0xDF && buf[3] === 0xA3) {
return {
ext: "webm",
mime: "video/webm"
};
}
if (buf[0] === 0x0 && buf[1] === 0x0 && buf[2] === 0x0 && buf[3] === 0x14 && buf[4] === 0x66 && buf[5] === 0x74 && buf[6] === 0x79 && buf[7] === 0x70) {
return {
ext: "mov",
mime: "video/quicktime"
};
}
if (buf[0] === 0x52 && buf[1] === 0x49 && buf[2] === 0x46 && buf[3] === 0x46 && buf[8] === 0x41 && buf[9] === 0x56 && buf[10] === 0x49) {
return {
ext: "avi",
mime: "video/x-msvideo"
};
}
if (buf[0] === 0x30 && buf[1] === 0x26 && buf[2] === 0xB2 && buf[3] === 0x75 && buf[4] === 0x8E && buf[5] === 0x66 && buf[6] === 0xCF && buf[7] === 0x11 && buf[8] === 0xA6 && buf[9] === 0xD9) {
return {
ext: "wmv",
mime: "video/x-ms-wmv"
};
}
if (buf[0] === 0x0 && buf[1] === 0x0 && buf[2] === 0x1 && buf[3].toString(16)[0] === "b") {
return {
ext: "mpg",
mime: "video/mpeg"
};
}
if ((buf[0] === 0x49 && buf[1] === 0x44 && buf[2] === 0x33) || (buf[0] === 0xFF && buf[1] === 0xfb)) {
return {
ext: "mp3",
mime: "audio/mpeg"
};
}
if ((buf[4] === 0x66 && buf[5] === 0x74 && buf[6] === 0x79 && buf[7] === 0x70 && buf[8] === 0x4D && buf[9] === 0x34 && buf[10] === 0x41) || (buf[0] === 0x4D && buf[1] === 0x34 && buf[2] === 0x41 && buf[3] === 0x20)) {
return {
ext: "m4a",
mime: "audio/m4a"
};
}
if (buf[0] === 0x4F && buf[1] === 0x67 && buf[2] === 0x67 && buf[3] === 0x53) {
return {
ext: "ogg",
mime: "audio/ogg"
};
}
if (buf[0] === 0x66 && buf[1] === 0x4C && buf[2] === 0x61 && buf[3] === 0x43) {
return {
ext: "flac",
mime: "audio/x-flac"
};
}
if (buf[0] === 0x52 && buf[1] === 0x49 && buf[2] === 0x46 && buf[3] === 0x46 && buf[8] === 0x57 && buf[9] === 0x41 && buf[10] === 0x56 && buf[11] === 0x45) {
return {
ext: "wav",
mime: "audio/x-wav"
};
}
if (buf[0] === 0x23 && buf[1] === 0x21 && buf[2] === 0x41 && buf[3] === 0x4D && buf[4] === 0x52 && buf[5] === 0x0A) {
return {
ext: "amr",
mime: "audio/amr"
};
}
if (buf[0] === 0x25 && buf[1] === 0x50 && buf[2] === 0x44 && buf[3] === 0x46) {
return {
ext: "pdf",
mime: "application/pdf"
};
}
if (buf[0] === 0x4D && buf[1] === 0x5A) {
return {
ext: "exe",
mime: "application/x-msdownload"
};
}
if ((buf[0] === 0x43 || buf[0] === 0x46) && buf[1] === 0x57 && buf[2] === 0x53) {
return {
ext: "swf",
mime: "application/x-shockwave-flash"
};
}
if (buf[0] === 0x7B && buf[1] === 0x5C && buf[2] === 0x72 && buf[3] === 0x74 && buf[4] === 0x66) {
return {
ext: "rtf",
mime: "application/rtf"
};
}
if (buf[0] === 0x77 && buf[1] === 0x4F && buf[2] === 0x46 && buf[3] === 0x46 && buf[4] === 0x00 && buf[5] === 0x01 && buf[6] === 0x00 && buf[7] === 0x00) {
return {
ext: "woff",
mime: "application/font-woff"
};
}
if (buf[0] === 0x77 && buf[1] === 0x4F && buf[2] === 0x46 && buf[3] === 0x32 && buf[4] === 0x00 && buf[5] === 0x01 && buf[6] === 0x00 && buf[7] === 0x00) {
return {
ext: "woff2",
mime: "application/font-woff"
};
}
if (buf[34] === 0x4C && buf[35] === 0x50 && ((buf[8] === 0x02 && buf[9] === 0x00 && buf[10] === 0x01) || (buf[8] === 0x01 && buf[9] === 0x00 && buf[10] === 0x00) || (buf[8] === 0x02 && buf[9] === 0x00 && buf[10] === 0x02))) {
return {
ext: "eot",
mime: "application/octet-stream"
};
}
if (buf[0] === 0x00 && buf[1] === 0x01 && buf[2] === 0x00 && buf[3] === 0x00 && buf[4] === 0x00) {
return {
ext: "ttf",
mime: "application/font-sfnt"
};
}
if (buf[0] === 0x4F && buf[1] === 0x54 && buf[2] === 0x54 && buf[3] === 0x4F && buf[4] === 0x00) {
return {
ext: "otf",
mime: "application/font-sfnt"
};
}
if (buf[0] === 0x00 && buf[1] === 0x00 && buf[2] === 0x01 && buf[3] === 0x00) {
return {
ext: "ico",
mime: "image/x-icon"
};
}
if (buf[0] === 0x46 && buf[1] === 0x4C && buf[2] === 0x56 && buf[3] === 0x01) {
return {
ext: "flv",
mime: "video/x-flv"
};
}
if (buf[0] === 0x25 && buf[1] === 0x21) {
return {
ext: "ps",
mime: "application/postscript"
};
}
if (buf[0] === 0xFD && buf[1] === 0x37 && buf[2] === 0x7A && buf[3] === 0x58 && buf[4] === 0x5A && buf[5] === 0x00) {
return {
ext: "xz",
mime: "application/x-xz"
};
}
if (buf[0] === 0x53 && buf[1] === 0x51 && buf[2] === 0x4C && buf[3] === 0x69) {
return {
ext: "sqlite",
mime: "application/x-sqlite3"
};
}
/**
*
* Added by n1474335 [n1474335@gmail.com] from here on
*
*/
if ((buf[0] === 0x1F && buf[1] === 0x9D) || (buf[0] === 0x1F && buf[1] === 0xA0)) {
return {
ext: "z, tar.z",
mime: "application/x-gtar"
};
}
if (buf[0] === 0x7F && buf[1] === 0x45 && buf[2] === 0x4C && buf[3] === 0x46) {
return {
ext: "none, axf, bin, elf, o, prx, puff, so",
mime: "application/x-executable",
desc: "Executable and Linkable Format file. No standard file extension."
};
}
if (buf[0] === 0xCA && buf[1] === 0xFE && buf[2] === 0xBA && buf[3] === 0xBE) {
return {
ext: "class",
mime: "application/java-vm"
};
}
if (buf[0] === 0xEF && buf[1] === 0xBB && buf[2] === 0xBF) {
return {
ext: "txt",
mime: "text/plain",
desc: "UTF-8 encoded Unicode byte order mark detected, commonly but not exclusively seen in text files."
};
}
// Must be before Little-endian UTF-16 BOM
if (buf[0] === 0xFF && buf[1] === 0xFE && buf[2] === 0x00 && buf[3] === 0x00) {
return {
ext: "UTF32LE",
mime: "charset/utf32le",
desc: "Little-endian UTF-32 encoded Unicode byte order mark detected."
};
}
if (buf[0] === 0xFF && buf[1] === 0xFE) {
return {
ext: "UTF16LE",
mime: "charset/utf16le",
desc: "Little-endian UTF-16 encoded Unicode byte order mark detected."
};
}
if ((buf[0x8001] === 0x43 && buf[0x8002] === 0x44 && buf[0x8003] === 0x30 && buf[0x8004] === 0x30 && buf[0x8005] === 0x31) ||
(buf[0x8801] === 0x43 && buf[0x8802] === 0x44 && buf[0x8803] === 0x30 && buf[0x8804] === 0x30 && buf[0x8805] === 0x31) ||
(buf[0x9001] === 0x43 && buf[0x9002] === 0x44 && buf[0x9003] === 0x30 && buf[0x9004] === 0x30 && buf[0x9005] === 0x31)) {
return {
ext: "iso",
mime: "application/octet-stream",
desc: "ISO 9660 CD/DVD image file"
};
}
if (buf[0] === 0xD0 && buf[1] === 0xCF && buf[2] === 0x11 && buf[3] === 0xE0 && buf[4] === 0xA1 && buf[5] === 0xB1 && buf[6] === 0x1A && buf[7] === 0xE1) {
return {
ext: "doc, xls, ppt",
mime: "application/msword, application/vnd.ms-excel, application/vnd.ms-powerpoint",
desc: "Microsoft Office documents"
};
}
if (buf[0] === 0x64 && buf[1] === 0x65 && buf[2] === 0x78 && buf[3] === 0x0A && buf[4] === 0x30 && buf[5] === 0x33 && buf[6] === 0x35 && buf[7] === 0x00) {
return {
ext: "dex",
mime: "application/octet-stream",
desc: "Dalvik Executable (Android)"
};
}
if (buf[0] === 0x4B && buf[1] === 0x44 && buf[2] === 0x4D) {
return {
ext: "vmdk",
mime: "application/vmdk, application/x-virtualbox-vmdk"
};
}
if (buf[0] === 0x43 && buf[1] === 0x72 && buf[2] === 0x32 && buf[3] === 0x34) {
return {
ext: "crx",
mime: "application/crx",
desc: "Google Chrome extension or packaged app"
};
}
if (buf[0] === 0x78 && (buf[1] === 0x01 || buf[1] === 0x9C || buf[1] === 0xDA || buf[1] === 0x5e)) {
return {
ext: "zlib",
mime: "application/x-deflate"
};
}
return null;
}
}
/**
* Byte frequencies of various languages generated from Wikipedia dumps taken in late 2017 and early 2018.
* The Chi-Squared test cannot accept expected values of 0, so 0.0001 has been used to account for bytes

5
src/core/lib/PGP.mjs
View file

@ -10,7 +10,8 @@
*
*/
import OperationError from "../errors/OperationError";
import OperationError from "../errors/OperationError.mjs";
import { isWorkerEnvironment } from "../Utils.mjs";
import kbpgp from "kbpgp";
import * as es6promisify from "es6-promisify";
const promisify = es6promisify.default ? es6promisify.default.promisify : es6promisify.promisify;
@ -45,7 +46,7 @@ export const ASP = kbpgp.ASP({
msg = `Stage: ${info.what}`;
}
if (ENVIRONMENT_IS_WORKER())
if (isWorkerEnvironment())
self.sendStatusMessage(msg);
}
});

285
src/core/lib/Protobuf.mjs Normal file
View file

@ -0,0 +1,285 @@
import Utils from "../Utils.mjs";
/**
* Protobuf lib. Contains functions to decode protobuf serialised
* data without a schema or .proto file.
*
* Provides utility functions to encode and decode variable length
* integers (varint).
*
* @author GCHQ Contributor [3]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
class Protobuf {
/**
* Protobuf constructor
*
* @param {byteArray|Uint8Array} data
*/
constructor(data) {
// Check we have a byteArray or Uint8Array
if (data instanceof Array || data instanceof Uint8Array) {
this.data = data;
} else {
throw new Error("Protobuf input must be a byteArray or Uint8Array");
}
// Set up masks
this.TYPE = 0x07;
this.NUMBER = 0x78;
this.MSB = 0x80;
this.VALUE = 0x7f;
// Declare offset and length
this.offset = 0;
this.LENGTH = data.length;
}
// Public Functions
/**
* Encode a varint from a number
*
* @param {number} number
* @returns {byteArray}
*/
static varIntEncode(number) {
const MSB = 0x80,
VALUE = 0x7f,
MSBALL = ~VALUE,
INT = Math.pow(2, 31);
const out = [];
let offset = 0;
while (number >= INT) {
out[offset++] = (number & 0xff) | MSB;
number /= 128;
}
while (number & MSBALL) {
out[offset++] = (number & 0xff) | MSB;
number >>>= 7;
}
out[offset] = number | 0;
return out;
}
/**
* Decode a varint from the byteArray
*
* @param {byteArray} input
* @returns {number}
*/
static varIntDecode(input) {
const pb = new Protobuf(input);
return pb._varInt();
}
/**
* Parse Protobuf data
*
* @param {byteArray} input
* @returns {Object}
*/
static decode(input) {
const pb = new Protobuf(input);
return pb._parse();
}
// Private Class Functions
/**
* Main private parsing function
*
* @private
* @returns {Object}
*/
_parse() {
let object = {};
// Continue reading whilst we still have data
while (this.offset < this.LENGTH) {
const field = this._parseField();
object = this._addField(field, object);
}
// Throw an error if we have gone beyond the end of the data
if (this.offset > this.LENGTH) {
throw new Error("Exhausted Buffer");
}
return object;
}
/**
* Add a field read from the protobuf data into the Object. As
* protobuf fields can appear multiple times, if the field already
* exists we need to add the new field into an array of fields
* for that key.
*
* @private
* @param {Object} field
* @param {Object} object
* @returns {Object}
*/
_addField(field, object) {
// Get the field key/values
const key = field.key;
const value = field.value;
object[key] = Object.prototype.hasOwnProperty.call(object, key) ?
object[key] instanceof Array ?
object[key].concat([value]) :
[object[key], value] :
value;
return object;
}
/**
* Parse a field and return the Object read from the record
*
* @private
* @returns {Object}
*/
_parseField() {
// Get the field headers
const header = this._fieldHeader();
const type = header.type;
const key = header.key;
switch (type) {
// varint
case 0:
return { "key": key, "value": this._varInt() };
// fixed 64
case 1:
return { "key": key, "value": this._uint64() };
// length delimited
case 2:
return { "key": key, "value": this._lenDelim() };
// fixed 32
case 5:
return { "key": key, "value": this._uint32() };
// unknown type
default:
throw new Error("Unknown type 0x" + type.toString(16));
}
}
/**
* Parse the field header and return the type and key
*
* @private
* @returns {Object}
*/
_fieldHeader() {
// Make sure we call type then number to preserve offset
return { "type": this._fieldType(), "key": this._fieldNumber() };
}
/**
* Parse the field type from the field header. Type is stored in the
* lower 3 bits of the tag byte. This does not move the offset on as
* we need to read the field number from the tag byte too.
*
* @private
* @returns {number}
*/
_fieldType() {
// Field type stored in lower 3 bits of tag byte
return this.data[this.offset] & this.TYPE;
}
/**
* Parse the field number (i.e. the key) from the field header. The
* field number is stored in the upper 5 bits of the tag byte - but
* is also varint encoded so the follow on bytes may need to be read
* when field numbers are > 15.
*
* @private
* @returns {number}
*/
_fieldNumber() {
let shift = -3;
let fieldNumber = 0;
do {
fieldNumber += shift < 28 ?
shift === -3 ?
(this.data[this.offset] & this.NUMBER) >> -shift :
(this.data[this.offset] & this.VALUE) << shift :
(this.data[this.offset] & this.VALUE) * Math.pow(2, shift);
shift += 7;
} while ((this.data[this.offset++] & this.MSB) === this.MSB);
return fieldNumber;
}
// Field Parsing Functions
/**
* Read off a varint from the data
*
* @private
* @returns {number}
*/
_varInt() {
let value = 0;
let shift = 0;
// Keep reading while upper bit set
do {
value += shift < 28 ?
(this.data[this.offset] & this.VALUE) << shift :
(this.data[this.offset] & this.VALUE) * Math.pow(2, shift);
shift += 7;
} while ((this.data[this.offset++] & this.MSB) === this.MSB);
return value;
}
/**
* Read off a 64 bit unsigned integer from the data
*
* @private
* @returns {number}
*/
_uint64() {
// Read off a Uint64
let num = this.data[this.offset++] * 0x1000000 + (this.data[this.offset++] << 16) + (this.data[this.offset++] << 8) + this.data[this.offset++];
num = num * 0x100000000 + this.data[this.offset++] * 0x1000000 + (this.data[this.offset++] << 16) + (this.data[this.offset++] << 8) + this.data[this.offset++];
return num;
}
/**
* Read off a length delimited field from the data
*
* @private
* @returns {Object|string}
*/
_lenDelim() {
// Read off the field length
const length = this._varInt();
const fieldBytes = this.data.slice(this.offset, this.offset + length);
let field;
try {
// Attempt to parse as a new Protobuf Object
const pbObject = new Protobuf(fieldBytes);
field = pbObject._parse();
} catch (err) {
// Otherwise treat as bytes
field = Utils.byteArrayToChars(fieldBytes);
}
// Move the offset and return the field
this.offset += length;
return field;
}
/**
* Read a 32 bit unsigned integer from the data
*
* @private
* @returns {number}
*/
_uint32() {
// Use a dataview to read off the integer
const dataview = new DataView(new Uint8Array(this.data.slice(this.offset, this.offset + 4)).buffer);
const value = dataview.getUint32(0);
this.offset += 4;
return value;
}
}
export default Protobuf;

2
src/core/lib/PublicKey.mjs
View file

@ -6,7 +6,7 @@
* @license Apache-2.0
*/
import { toHex, fromHex } from "./Hex";
import { toHex, fromHex } from "./Hex.mjs";
/**
* Formats Distinguished Name (DN) strings.

93
src/core/lib/QRCode.mjs Normal file
View file

@ -0,0 +1,93 @@
/**
* QR code resources
*
* @author j433866 [j433866@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
import jsQR from "jsqr";
import qr from "qr-image";
import jimp from "jimp";
import Utils from "../Utils.mjs";
/**
* Parses a QR code image from an image
*
* @param {ArrayBuffer} input
* @param {boolean} normalise
* @returns {string}
*/
export async function parseQrCode(input, normalise) {
let image;
try {
image = await jimp.read(input);
} catch (err) {
throw new OperationError(`Error opening image. (${err})`);
}
try {
if (normalise) {
image.rgba(false);
image.background(0xFFFFFFFF);
image.normalize();
image.greyscale();
image = await image.getBufferAsync(jimp.MIME_JPEG);
image = await jimp.read(image);
}
} catch (err) {
throw new OperationError(`Error normalising iamge. (${err})`);
}
const qrData = jsQR(image.bitmap.data, image.getWidth(), image.getHeight());
if (qrData) {
return qrData.data;
} else {
throw new OperationError("Could not read a QR code from the image.");
}
}
/**
* Generates a QR code from the input string
*
* @param {string} input
* @param {string} format
* @param {number} moduleSize
* @param {number} margin
* @param {string} errorCorrection
* @returns {ArrayBuffer}
*/
export function generateQrCode(input, format, moduleSize, margin, errorCorrection) {
const formats = ["SVG", "EPS", "PDF", "PNG"];
if (!formats.includes(format.toUpperCase())) {
throw new OperationError("Unsupported QR code format.");
}
let qrImage;
try {
qrImage = qr.imageSync(input, {
type: format,
size: moduleSize,
margin: margin,
"ec_level": errorCorrection.charAt(0).toUpperCase()
});
} catch (err) {
throw new OperationError(`Error generating QR code. (${err})`);
}
if (!qrImage) {
throw new OperationError("Error generating QR code.");
}
switch (format) {
case "SVG":
case "EPS":
case "PDF":
return Utils.strToArrayBuffer(qrImage);
case "PNG":
return qrImage.buffer;
default:
throw new OperationError("Unsupported QR code format.");
}
}

263
src/core/lib/Stream.mjs Normal file
View file

@ -0,0 +1,263 @@
/**
* Stream class for parsing binary protocols.
*
* @author n1474335 [n1474335@gmail.com]
* @author tlwr [toby@toby.codes]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*
*/
/**
* A Stream can be used to traverse a binary blob, interpreting sections of it
* as various data types.
*/
export default class Stream {
/**
* Stream constructor.
*
* @param {Uint8Array} input
*/
constructor(input) {
this.bytes = input;
this.length = this.bytes.length;
this.position = 0;
this.bitPos = 0;
}
/**
* Get a number of bytes from the current position.
*
* @param {number} numBytes
* @returns {Uint8Array}
*/
getBytes(numBytes) {
if (this.position > this.length) return undefined;
const newPosition = this.position + numBytes;
const bytes = this.bytes.slice(this.position, newPosition);
this.position = newPosition;
this.bitPos = 0;
return bytes;
}
/**
* Interpret the following bytes as a string, stopping at the next null byte or
* the supplied limit.
*
* @param {number} numBytes
* @returns {string}
*/
readString(numBytes) {
if (this.position > this.length) return undefined;
let result = "";
for (let i = this.position; i < this.position + numBytes; i++) {
const currentByte = this.bytes[i];
if (currentByte === 0) break;
result += String.fromCharCode(currentByte);
}
this.position += numBytes;
this.bitPos = 0;
return result;
}
/**
* Interpret the following bytes as an integer in big or little endian.
*
* @param {number} numBytes
* @param {string} [endianness="be"]
* @returns {number}
*/
readInt(numBytes, endianness="be") {
if (this.position > this.length) return undefined;
let val = 0;
if (endianness === "be") {
for (let i = this.position; i < this.position + numBytes; i++) {
val = val << 8;
val |= this.bytes[i];
}
} else {
for (let i = this.position + numBytes - 1; i >= this.position; i--) {
val = val << 8;
val |= this.bytes[i];
}
}
this.position += numBytes;
this.bitPos = 0;
return val;
}
/**
* Reads a number of bits from the buffer.
*
* @TODO Add endianness
*
* @param {number} numBits
* @returns {number}
*/
readBits(numBits) {
if (this.position > this.length) return undefined;
let bitBuf = 0,
bitBufLen = 0;
// Add remaining bits from current byte
bitBuf = (this.bytes[this.position++] & bitMask(this.bitPos)) >>> this.bitPos;
bitBufLen = 8 - this.bitPos;
this.bitPos = 0;
// Not enough bits yet
while (bitBufLen < numBits) {
bitBuf |= this.bytes[this.position++] << bitBufLen;
bitBufLen += 8;
}
// Reverse back to numBits
if (bitBufLen > numBits) {
const excess = bitBufLen - numBits;
bitBuf &= (1 << numBits) - 1;
bitBufLen -= excess;
this.position--;
this.bitPos = 8 - excess;
}
return bitBuf;
/**
* Calculates the bit mask based on the current bit position.
*
* @param {number} bitPos
* @returns {number} The bit mask
*/
function bitMask(bitPos) {
return 256 - (1 << bitPos);
}
}
/**
* Consume the stream until we reach the specified byte or sequence of bytes.
*
* @param {number|List<number>} val
*/
continueUntil(val) {
if (this.position > this.length) return;
this.bitPos = 0;
if (typeof val === "number") {
while (++this.position < this.length && this.bytes[this.position] !== val) {
continue;
}
return;
}
// val is an array
let found = false;
while (!found && this.position < this.length) {
while (++this.position < this.length && this.bytes[this.position] !== val[0]) {
continue;
}
found = true;
for (let i = 1; i < val.length; i++) {
if (this.position + i > this.length || this.bytes[this.position + i] !== val[i])
found = false;
}
}
}
/**
* Consume the next byte if it matches the supplied value.
*
* @param {number} val
*/
consumeIf(val) {
if (this.bytes[this.position] === val) {
this.position++;
this.bitPos = 0;
}
}
/**
* Move forwards through the stream by the specified number of bytes.
*
* @param {number} numBytes
*/
moveForwardsBy(numBytes) {
const pos = this.position + numBytes;
if (pos < 0 || pos > this.length)
throw new Error("Cannot move to position " + pos + " in stream. Out of bounds.");
this.position = pos;
this.bitPos = 0;
}
/**
* Move backwards through the stream by the specified number of bytes.
*
* @param {number} numBytes
*/
moveBackwardsBy(numBytes) {
const pos = this.position - numBytes;
if (pos < 0 || pos > this.length)
throw new Error("Cannot move to position " + pos + " in stream. Out of bounds.");
this.position = pos;
this.bitPos = 0;
}
/**
* Move backwards through the strem by the specified number of bits.
*
* @param {number} numBits
*/
moveBackwardsByBits(numBits) {
if (numBits <= this.bitPos) {
this.bitPos -= numBits;
} else {
if (this.bitPos > 0) {
numBits -= this.bitPos;
this.bitPos = 0;
}
while (numBits > 0) {
this.moveBackwardsBy(1);
this.bitPos = 8;
this.moveBackwardsByBits(numBits);
numBits -= 8;
}
}
}
/**
* Move to a specified position in the stream.
*
* @param {number} pos
*/
moveTo(pos) {
if (pos < 0 || pos > this.length)
throw new Error("Cannot move to position " + pos + " in stream. Out of bounds.");
this.position = pos;
this.bitPos = 0;
}
/**
* Returns true if there are more bytes left in the stream.
*
* @returns {boolean}
*/
hasMore() {
return this.position < this.length;
}
/**
* Returns a slice of the stream up to the current position.
*
* @returns {Uint8Array}
*/
carve() {
if (this.bitPos > 0) this.position++;
return this.bytes.slice(0, this.position);
}
}

78
src/core/lib/TLVParser.mjs Normal file
View file

@ -0,0 +1,78 @@
/**
* Parser for Type-length-value data.
*
* @author gchq77703 []
* @author n1474335 [n1474335@gmail.com]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
const defaults = {
location: 0,
bytesInLength: 1,
basicEncodingRules: false
};
/**
* TLVParser library
*/
export default class TLVParser {
/**
* TLVParser constructor
*
* @param {byteArray|Uint8Array} input
* @param {Object} options
*/
constructor(input, options) {
this.input = input;
Object.assign(this, defaults, options);
}
/**
* @returns {number}
*/
getLength() {
if (this.basicEncodingRules) {
const bit = this.input[this.location];
if (bit & 0x80) {
this.bytesInLength = bit & ~0x80;
} else {
this.location++;
return bit & ~0x80;
}
}
let length = 0;
for (let i = 0; i < this.bytesInLength; i++) {
length += this.input[this.location] * Math.pow(Math.pow(2, 8), i);
this.location++;
}
return length;
}
/**
* @param {number} length
* @returns {number[]}
*/
getValue(length) {
const value = [];
for (let i = 0; i < length; i++) {
if (this.location > this.input.length) return value;
value.push(this.input[this.location]);
this.location++;
}
return value;
}
/**
* @returns {boolean}
*/
atEnd() {
return this.input.length <= this.location;
}
}

227
src/core/lib/Typex.mjs Normal file
View file

@ -0,0 +1,227 @@
/**
* Emulation of the Typex machine.
*
* @author s2224834
* @author The National Museum of Computing - Bombe Rebuild Project
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import OperationError from "../errors/OperationError.mjs";
import * as Enigma from "../lib/Enigma.mjs";
import Utils from "../Utils.mjs";
/**
* A set of example Typex rotors. No Typex rotor wirings are publicly available, so these are
* randomised.
*/
export const ROTORS = [
{name: "Example 1", value: "MCYLPQUVRXGSAOWNBJEZDTFKHI<BFHNQUW"},
{name: "Example 2", value: "KHWENRCBISXJQGOFMAPVYZDLTU<BFHNQUW"},
{name: "Example 3", value: "BYPDZMGIKQCUSATREHOJNLFWXV<BFHNQUW"},
{name: "Example 4", value: "ZANJCGDLVHIXOBRPMSWQUKFYET<BFHNQUW"},
{name: "Example 5", value: "QXBGUTOVFCZPJIHSWERYNDAMLK<BFHNQUW"},
{name: "Example 6", value: "BDCNWUEIQVFTSXALOGZJYMHKPR<BFHNQUW"},
{name: "Example 7", value: "WJUKEIABMSGFTQZVCNPHORDXYL<BFHNQUW"},
{name: "Example 8", value: "TNVCZXDIPFWQKHSJMAOYLEURGB<BFHNQUW"},
];
/**
* An example Typex reflector. Again, randomised.
*/
export const REFLECTORS = [
{name: "Example", value: "AN BC FG IE KD LU MH OR TS VZ WQ XJ YP"},
];
// Special character handling on Typex keyboard
const KEYBOARD = {
"Q": "1", "W": "2", "E": "3", "R": "4", "T": "5", "Y": "6", "U": "7", "I": "8", "O": "9", "P": "0",
"A": "-", "S": "/", "D": "Z", "F": "%", "G": "X", "H": "£", "K": "(", "L": ")",
"C": "V", "B": "'", "N": ",", "M": "."
};
const KEYBOARD_REV = {};
for (const i of Object.keys(KEYBOARD)) {
KEYBOARD_REV[KEYBOARD[i]] = i;
}
/**
* Typex machine. A lot like the Enigma, but five rotors, of which the first two are static.
*/
export class TypexMachine extends Enigma.EnigmaBase {
/**
* TypexMachine constructor.
*
* @param {Object[]} rotors - List of Rotors.
* @param {Object} reflector - A Reflector.
* @param {Plugboard} plugboard - A Plugboard.
*/
constructor(rotors, reflector, plugboard, keyboard) {
super(rotors, reflector, plugboard);
if (rotors.length !== 5) {
throw new OperationError("Typex must have 5 rotors");
}
this.keyboard = keyboard;
}
/**
* This is the same as the Enigma step function, it's just that the right-
* most two rotors are static.
*/
step() {
const r0 = this.rotors[2];
const r1 = this.rotors[3];
r0.step();
// The second test here is the double-stepping anomaly
if (r0.steps.has(r0.pos) || r1.steps.has(Utils.mod(r1.pos + 1, 26))) {
r1.step();
if (r1.steps.has(r1.pos)) {
const r2 = this.rotors[4];
r2.step();
}
}
}
/**
* Encrypt/decrypt data. This is identical to the Enigma version cryptographically, but we have
* additional handling for the Typex's keyboard (which handles various special characters by
* mapping them to particular letter combinations).
*
* @param {string} input - The data to encrypt/decrypt.
* @return {string}
*/
crypt(input) {
let inputMod = input;
if (this.keyboard === "Encrypt") {
inputMod = "";
// true = in symbol mode
let mode = false;
for (const x of input) {
if (x === " ") {
inputMod += "X";
} else if (mode) {
if (Object.prototype.hasOwnProperty.call(KEYBOARD_REV, x)) {
inputMod += KEYBOARD_REV[x];
} else {
mode = false;
inputMod += "V" + x;
}
} else {
if (Object.prototype.hasOwnProperty.call(KEYBOARD_REV, x)) {
mode = true;
inputMod += "Z" + KEYBOARD_REV[x];
} else {
inputMod += x;
}
}
}
}
const output = super.crypt(inputMod);
let outputMod = output;
if (this.keyboard === "Decrypt") {
outputMod = "";
let mode = false;
for (const x of output) {
if (x === "X") {
outputMod += " ";
} else if (x === "V") {
mode = false;
} else if (x === "Z") {
mode = true;
} else if (mode) {
outputMod += KEYBOARD[x];
} else {
outputMod += x;
}
}
}
return outputMod;
}
}
/**
* Typex rotor. Like an Enigma rotor, but no ring setting, and can be reversed.
*/
export class Rotor extends Enigma.Rotor {
/**
* Rotor constructor.
*
* @param {string} wiring - A 26 character string of the wiring order.
* @param {string} steps - A 0..26 character string of stepping points.
* @param {bool} reversed - Whether to reverse the rotor.
* @param {char} ringSetting - Ring setting of the rotor.
* @param {char} initialPosition - The initial position of the rotor.
*/
constructor(wiring, steps, reversed, ringSetting, initialPos) {
let wiringMod = wiring;
if (reversed) {
const outMap = new Array(26);
for (let i=0; i<26; i++) {
// wiring[i] is the original output
// Enigma.LETTERS[i] is the original input
const input = Utils.mod(26 - Enigma.a2i(wiring[i]), 26);
const output = Enigma.i2a(Utils.mod(26 - Enigma.a2i(Enigma.LETTERS[i]), 26));
outMap[input] = output;
}
wiringMod = outMap.join("");
}
super(wiringMod, steps, ringSetting, initialPos);
}
}
/**
* Typex input plugboard. Based on a Rotor, because it allows arbitrary maps, not just switches
* like the Enigma plugboard.
* Not to be confused with the reflector plugboard.
* This is also where the Typex's backwards input wiring is implemented - it's a bit of a hack, but
* it means everything else continues to work like in the Enigma.
*/
export class Plugboard extends Enigma.Rotor {
/**
* Typex plugboard constructor.
*
* @param {string} wiring - 26 character string of mappings from A-Z, as per rotors, or "".
*/
constructor(wiring) {
// Typex input wiring is backwards vs Enigma: that is, letters enter the rotors in a
// clockwise order, vs. Enigma's anticlockwise (or vice versa depending on which side
// you're looking at it from). I'm doing the transform here to avoid having to rewrite
// the Engima crypt() method in Typex as well.
// Note that the wiring for the reflector is the same way around as Enigma, so no
// transformation is necessary on that side.
// We're going to achieve this by mapping the plugboard settings through an additional
// transform that mirrors the alphabet before we pass it to the superclass.
if (!/^[A-Z]{26}$/.test(wiring)) {
throw new OperationError("Plugboard wiring must be 26 unique uppercase letters");
}
const reversed = "AZYXWVUTSRQPONMLKJIHGFEDCB";
wiring = wiring.replace(/./g, x => {
return reversed[Enigma.a2i(x)];
});
try {
super(wiring, "", "A", "A");
} catch (err) {
throw new OperationError(err.message.replace("Rotor", "Plugboard"));
}
}
/**
* Transform a character through this rotor forwards.
*
* @param {number} c - The character.
* @returns {number}
*/
transform(c) {
return Utils.mod(this.map[Utils.mod(c + this.pos, 26)] - this.pos, 26);
}
/**
* Transform a character through this rotor backwards.
*
* @param {number} c - The character.
* @returns {number}
*/
revTransform(c) {
return Utils.mod(this.revMap[Utils.mod(c + this.pos, 26)] - this.pos, 26);
}
}

2
src/core/lib/Zlib.mjs
View file

@ -6,7 +6,7 @@
* @license Apache-2.0
*/
import zlibAndGzip from "zlibjs/bin/zlib_and_gzip.min";
import zlibAndGzip from "zlibjs/bin/zlib_and_gzip.min.js";
const Zlib = zlibAndGzip.Zlib;

63
src/core/operations/A1Z26CipherDecode.mjs Normal file
View file

@ -0,0 +1,63 @@
/**
* @author Jarmo van Lenthe [github.com/jarmovanlenthe]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import {DELIM_OPTIONS} from "../lib/Delim.mjs";
import OperationError from "../errors/OperationError.mjs";
/**
* A1Z26 Cipher Decode operation
*/
class A1Z26CipherDecode extends Operation {
/**
* A1Z26CipherDecode constructor
*/
constructor() {
super();
this.name = "A1Z26 Cipher Decode";
this.module = "Ciphers";
this.description = "Converts alphabet order numbers into their corresponding alphabet character.<br><br>e.g. <code>1</code> becomes <code>a</code> and <code>2</code> becomes <code>b</code>.";
this.infoURL = "";
this.inputType = "string";
this.outputType = "string";
this.args = [
{
name: "Delimiter",
type: "option",
value: DELIM_OPTIONS
}
];
}
/**
* @param {string} input
* @param {Object[]} args
* @returns {string}
*/
run(input, args) {
const delim = Utils.charRep(args[0] || "Space");
if (input.length === 0) {
return [];
}
const bites = input.split(delim);
let latin1 = "";
for (let i = 0; i < bites.length; i++) {
if (bites[i] < 1 || bites[i] > 26) {
throw new OperationError("Error: all numbers must be between 1 and 26.");
}
latin1 += Utils.chr(parseInt(bites[i], 10) + 96);
}
return latin1;
}
}
export default A1Z26CipherDecode;

61
src/core/operations/A1Z26CipherEncode.mjs Normal file
View file

@ -0,0 +1,61 @@
/**
* @author Jarmo van Lenthe [github.com/jarmovanlenthe]
* @copyright Crown Copyright 2018
* @license Apache-2.0
*/
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import {DELIM_OPTIONS} from "../lib/Delim.mjs";
/**
* A1Z26 Cipher Encode operation
*/
class A1Z26CipherEncode extends Operation {
/**
* A1Z26CipherEncode constructor
*/
constructor() {
super();
this.name = "A1Z26 Cipher Encode";
this.module = "Ciphers";
this.description = "Converts alphabet characters into their corresponding alphabet order number.<br><br>e.g. <code>a</code> becomes <code>1</code> and <code>b</code> becomes <code>2</code>.<br><br>Non-alphabet characters are dropped.";
this.infoURL = "";
this.inputType = "string";
this.outputType = "string";
this.args = [
{
name: "Delimiter",
type: "option",
value: DELIM_OPTIONS
}
];
}
/**
* @param {string} input
* @param {Object[]} args
* @returns {string}
*/
run(input, args) {
const delim = Utils.charRep(args[0] || "Space");
let output = "";
const sanitizedinput = input.toLowerCase(),
charcode = Utils.strToCharcode(sanitizedinput);
for (let i = 0; i < charcode.length; i++) {
const ordinal = charcode[i] - 96;
if (ordinal > 0 && ordinal <= 26) {
output += ordinal.toString(10) + delim;
}
}
return output.slice(0, -delim.length);
}
}
export default A1Z26CipherEncode;

8
src/core/operations/ADD.mjs
View file

@ -4,9 +4,9 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Utils from "../Utils";
import { bitOp, add } from "../lib/BitwiseOp";
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import { bitOp, add, BITWISE_OP_DELIMS } from "../lib/BitwiseOp.mjs";
/**
* ADD operation
@ -30,7 +30,7 @@ class ADD extends Operation {
"name": "Key",
"type": "toggleString",
"value": "",
"toggleValues": ["Hex", "Decimal", "Base64", "UTF8", "Latin1"]
"toggleValues": BITWISE_OP_DELIMS
}
];
}

12
src/core/operations/AESDecrypt.mjs
View file

@ -4,10 +4,10 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Utils from "../Utils";
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import forge from "node-forge/dist/forge.min.js";
import OperationError from "../errors/OperationError";
import OperationError from "../errors/OperationError.mjs";
/**
* AES Decrypt operation
@ -71,8 +71,8 @@ class AESDecrypt extends Operation {
* @throws {OperationError} if cannot decrypt input or invalid key length
*/
run(input, args) {
const key = Utils.convertToByteArray(args[0].string, args[0].option),
iv = Utils.convertToByteArray(args[1].string, args[1].option),
const key = Utils.convertToByteString(args[0].string, args[0].option),
iv = Utils.convertToByteString(args[1].string, args[1].option),
mode = args[2],
inputType = args[3],
outputType = args[4],
@ -91,7 +91,7 @@ The following algorithms will be used based on the size of the key:
const decipher = forge.cipher.createDecipher("AES-" + mode, key);
decipher.start({
iv: iv,
iv: iv.length === 0 ? "" : iv,
tag: gcmTag
});
decipher.update(forge.util.createBuffer(input));

10
src/core/operations/AESEncrypt.mjs
View file

@ -4,10 +4,10 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Utils from "../Utils";
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import forge from "node-forge/dist/forge.min.js";
import OperationError from "../errors/OperationError";
import OperationError from "../errors/OperationError.mjs";
/**
* AES Encrypt operation
@ -65,8 +65,8 @@ class AESEncrypt extends Operation {
* @throws {OperationError} if invalid key length
*/
run(input, args) {
const key = Utils.convertToByteArray(args[0].string, args[0].option),
iv = Utils.convertToByteArray(args[1].string, args[1].option),
const key = Utils.convertToByteString(args[0].string, args[0].option),
iv = Utils.convertToByteString(args[1].string, args[1].option),
mode = args[2],
inputType = args[3],
outputType = args[4];

8
src/core/operations/AND.mjs
View file

@ -4,9 +4,9 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Utils from "../Utils";
import { bitOp, and } from "../lib/BitwiseOp";
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import { bitOp, and, BITWISE_OP_DELIMS } from "../lib/BitwiseOp.mjs";
/**
* AND operation
@ -30,7 +30,7 @@ class AND extends Operation {
"name": "Key",
"type": "toggleString",
"value": "",
"toggleValues": ["Hex", "Decimal", "Base64", "UTF8", "Latin1"]
"toggleValues": BITWISE_OP_DELIMS
}
];
}

2
src/core/operations/AddLineNumbers.mjs
View file

@ -4,7 +4,7 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Operation from "../Operation.mjs";
/**
* Add line numbers operation

267
src/core/operations/AddTextToImage.mjs Normal file
View file

@ -0,0 +1,267 @@
/**
* @author j433866 [j433866@gmail.com]
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import Operation from "../Operation.mjs";
import OperationError from "../errors/OperationError.mjs";
import { isImage } from "../lib/FileType.mjs";
import { toBase64 } from "../lib/Base64.mjs";
import { isWorkerEnvironment } from "../Utils.mjs";
import jimp from "jimp";
/**
* Add Text To Image operation
*/
class AddTextToImage extends Operation {
/**
* AddTextToImage constructor
*/
constructor() {
super();
this.name = "Add Text To Image";
this.module = "Image";
this.description = "Adds text onto an image.<br><br>Text can be horizontally or vertically aligned, or the position can be manually specified.<br>Variants of the Roboto font face are available in any size or colour.";
this.infoURL = "";
this.inputType = "ArrayBuffer";
this.outputType = "ArrayBuffer";
this.presentType = "html";
this.args = [
{
name: "Text",
type: "string",
value: ""
},
{
name: "Horizontal align",
type: "option",
value: ["None", "Left", "Center", "Right"]
},
{
name: "Vertical align",
type: "option",
value: ["None", "Top", "Middle", "Bottom"]
},
{
name: "X position",
type: "number",
value: 0
},
{
name: "Y position",
type: "number",
value: 0
},
{
name: "Size",
type: "number",
value: 32,
min: 8
},
{
name: "Font face",
type: "option",
value: [
"Roboto",
"Roboto Black",
"Roboto Mono",
"Roboto Slab"
]
},
{
name: "Red",
type: "number",
value: 255,
min: 0,
max: 255
},
{
name: "Green",
type: "number",
value: 255,
min: 0,
max: 255
},
{
name: "Blue",
type: "number",
value: 255,
min: 0,
max: 255
},
{
name: "Alpha",
type: "number",
value: 255,
min: 0,
max: 255
}
];
}
/**
* @param {ArrayBuffer} input
* @param {Object[]} args
* @returns {byteArray}
*/
async run(input, args) {
const text = args[0],
hAlign = args[1],
vAlign = args[2],
size = args[5],
fontFace = args[6],
red = args[7],
green = args[8],
blue = args[9],
alpha = args[10];
let xPos = args[3],
yPos = args[4];
if (!isImage(input)) {
throw new OperationError("Invalid file type.");
}
let image;
try {
image = await jimp.read(input);
} catch (err) {
throw new OperationError(`Error loading image. (${err})`);
}
try {
if (isWorkerEnvironment())
self.sendStatusMessage("Adding text to image...");
const fontsMap = {};
const fonts = [
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/Roboto72White.fnt"),
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/RobotoBlack72White.fnt"),
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/RobotoMono72White.fnt"),
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/RobotoSlab72White.fnt")
];
await Promise.all(fonts)
.then(fonts => {
fontsMap.Roboto = fonts[0];
fontsMap["Roboto Black"] = fonts[1];
fontsMap["Roboto Mono"] = fonts[2];
fontsMap["Roboto Slab"] = fonts[3];
});
// Make Webpack load the png font images
await Promise.all([
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/Roboto72White.png"),
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/RobotoSlab72White.png"),
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/RobotoMono72White.png"),
import(/* webpackMode: "eager" */ "../../web/static/fonts/bmfonts/RobotoBlack72White.png")
]);
const font = fontsMap[fontFace];
// LoadFont needs an absolute url, so append the font name to self.docURL
const jimpFont = await jimp.loadFont(self.docURL + "/" + font.default);
jimpFont.pages.forEach(function(page) {
if (page.bitmap) {
// Adjust the RGB values of the image pages to change the font colour.
const pageWidth = page.bitmap.width;
const pageHeight = page.bitmap.height;
for (let ix = 0; ix < pageWidth; ix++) {
for (let iy = 0; iy < pageHeight; iy++) {
const idx = (iy * pageWidth + ix) << 2;
const newRed = page.bitmap.data[idx] - (255 - red);
const newGreen = page.bitmap.data[idx + 1] - (255 - green);
const newBlue = page.bitmap.data[idx + 2] - (255 - blue);
const newAlpha = page.bitmap.data[idx + 3] - (255 - alpha);
// Make sure the bitmap values don't go below 0 as that makes jimp very unhappy
page.bitmap.data[idx] = (newRed > 0) ? newRed : 0;
page.bitmap.data[idx + 1] = (newGreen > 0) ? newGreen : 0;
page.bitmap.data[idx + 2] = (newBlue > 0) ? newBlue : 0;
page.bitmap.data[idx + 3] = (newAlpha > 0) ? newAlpha : 0;
}
}
}
});
// Create a temporary image to hold the rendered text
const textImage = new jimp(jimp.measureText(jimpFont, text), jimp.measureTextHeight(jimpFont, text));
textImage.print(jimpFont, 0, 0, text);
// Scale the rendered text image to the correct size
const scaleFactor = size / 72;
if (size !== 1) {
// Use bicubic for decreasing size
if (size > 1) {
textImage.scale(scaleFactor, jimp.RESIZE_BICUBIC);
} else {
textImage.scale(scaleFactor, jimp.RESIZE_BILINEAR);
}
}
// If using the alignment options, calculate the pixel values AFTER the image has been scaled
switch (hAlign) {
case "Left":
xPos = 0;
break;
case "Center":
xPos = (image.getWidth() / 2) - (textImage.getWidth() / 2);
break;
case "Right":
xPos = image.getWidth() - textImage.getWidth();
break;
}
switch (vAlign) {
case "Top":
yPos = 0;
break;
case "Middle":
yPos = (image.getHeight() / 2) - (textImage.getHeight() / 2);
break;
case "Bottom":
yPos = image.getHeight() - textImage.getHeight();
break;
}
// Blit the rendered text image onto the original source image
image.blit(textImage, xPos, yPos);
let imageBuffer;
if (image.getMIME() === "image/gif") {
imageBuffer = await image.getBufferAsync(jimp.MIME_PNG);
} else {
imageBuffer = await image.getBufferAsync(jimp.AUTO);
}
return imageBuffer.buffer;
} catch (err) {
throw new OperationError(`Error adding text to image. (${err})`);
}
}
/**
* Displays the blurred image using HTML for web apps
*
* @param {ArrayBuffer} data
* @returns {html}
*/
present(data) {
if (!data.byteLength) return "";
const dataArray = new Uint8Array(data);
const type = isImage(dataArray);
if (!type) {
throw new OperationError("Invalid file type.");
}
return `<img src="data:${type};base64,${toBase64(dataArray)}">`;
}
}
export default AddTextToImage;

11
src/core/operations/Adler32Checksum.mjs
View file

@ -4,8 +4,8 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Utils from "../Utils";
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
/**
* Adler-32 Checksum operation
@ -19,16 +19,16 @@ class Adler32Checksum extends Operation {
super();
this.name = "Adler-32 Checksum";
this.module = "Hashing";
this.module = "Crypto";
this.description = "Adler-32 is a checksum algorithm which was invented by Mark Adler in 1995, and is a modification of the Fletcher checksum. Compared to a cyclic redundancy check of the same length, it trades reliability for speed (preferring the latter).<br><br>Adler-32 is more reliable than Fletcher-16, and slightly less reliable than Fletcher-32.";
this.infoURL = "https://wikipedia.org/wiki/Adler-32";
this.inputType = "byteArray";
this.inputType = "ArrayBuffer";
this.outputType = "string";
this.args = [];
}
/**
* @param {byteArray} input
* @param {ArrayBuffer} input
* @param {Object[]} args
* @returns {string}
*/
@ -36,6 +36,7 @@ class Adler32Checksum extends Operation {
const MOD_ADLER = 65521;
let a = 1,
b = 0;
input = new Uint8Array(input);
for (let i = 0; i < input.length; i++) {
a += input[i];

6
src/core/operations/AffineCipherDecode.mjs
View file

@ -4,9 +4,9 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Utils from "../Utils";
import OperationError from "../errors/OperationError";
import Operation from "../Operation.mjs";
import Utils from "../Utils.mjs";
import OperationError from "../errors/OperationError.mjs";
/**
* Affine Cipher Decode operation

4
src/core/operations/AffineCipherEncode.mjs
View file

@ -4,8 +4,8 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import { affineEncode } from "../lib/Ciphers";
import Operation from "../Operation.mjs";
import { affineEncode } from "../lib/Ciphers.mjs";
/**
* Affine Cipher Encode operation

6
src/core/operations/AnalyseHash.mjs
View file

@ -4,8 +4,8 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import OperationError from "../errors/OperationError";
import Operation from "../Operation.mjs";
import OperationError from "../errors/OperationError.mjs";
/**
* Analyse hash operation
@ -19,7 +19,7 @@ class AnalyseHash extends Operation {
super();
this.name = "Analyse hash";
this.module = "Hashing";
this.module = "Crypto";
this.description = "Tries to determine information about a given hash and suggests which algorithm may have been used to generate it based on its length.";
this.infoURL = "https://wikipedia.org/wiki/Comparison_of_cryptographic_hash_functions";
this.inputType = "string";

4
src/core/operations/AtbashCipher.mjs
View file

@ -4,8 +4,8 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import { affineEncode } from "../lib/Ciphers";
import Operation from "../Operation.mjs";
import { affineEncode } from "../lib/Ciphers.mjs";
/**
* Atbash Cipher operation

79
src/core/operations/BLAKE2b.mjs Normal file
View file

@ -0,0 +1,79 @@
/**
* @author h345983745
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import Operation from "../Operation.mjs";
import blakejs from "blakejs";
import OperationError from "../errors/OperationError.mjs";
import Utils from "../Utils.mjs";
import { toBase64 } from "../lib/Base64.mjs";
/**
* BLAKE2b operation
*/
class BLAKE2b extends Operation {
/**
* BLAKE2b constructor
*/
constructor() {
super();
this.name = "BLAKE2b";
this.module = "Hashing";
this.description = `Performs BLAKE2b hashing on the input.
<br><br> BLAKE2b is a flavour of the BLAKE cryptographic hash function that is optimized for 64-bit platforms and produces digests of any size between 1 and 64 bytes.
<br><br> Supports the use of an optional key.`;
this.infoURL = "https://wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE2b_algorithm";
this.inputType = "ArrayBuffer";
this.outputType = "string";
this.args = [
{
"name": "Size",
"type": "option",
"value": ["512", "384", "256", "160", "128"]
}, {
"name": "Output Encoding",
"type": "option",
"value": ["Hex", "Base64", "Raw"]
}, {
"name": "Key",
"type": "toggleString",
"value": "",
"toggleValues": ["UTF8", "Decimal", "Base64", "Hex", "Latin1"]
}
];
}
/**
* @param {ArrayBuffer} input
* @param {Object[]} args
* @returns {string} The input having been hashed with BLAKE2b in the encoding format speicifed.
*/
run(input, args) {
const [outSize, outFormat] = args;
let key = Utils.convertToByteArray(args[2].string || "", args[2].option);
if (key.length === 0) {
key = null;
} else if (key.length > 64) {
throw new OperationError(["Key cannot be greater than 64 bytes", "It is currently " + key.length + " bytes."].join("\n"));
}
input = new Uint8Array(input);
switch (outFormat) {
case "Hex":
return blakejs.blake2bHex(input, key, outSize / 8);
case "Base64":
return toBase64(blakejs.blake2b(input, key, outSize / 8));
case "Raw":
return Utils.arrayBufferToStr(blakejs.blake2b(input, key, outSize / 8).buffer);
default:
return new OperationError("Unsupported Output Type");
}
}
}
export default BLAKE2b;

80
src/core/operations/BLAKE2s.mjs Normal file
View file

@ -0,0 +1,80 @@
/**
* @author h345983745
* @copyright Crown Copyright 2019
* @license Apache-2.0
*/
import Operation from "../Operation.mjs";
import blakejs from "blakejs";
import OperationError from "../errors/OperationError.mjs";
import Utils from "../Utils.mjs";
import { toBase64 } from "../lib/Base64.mjs";
/**
* BLAKE2s Operation
*/
class BLAKE2s extends Operation {
/**
* BLAKE2s constructor
*/
constructor() {
super();
this.name = "BLAKE2s";
this.module = "Hashing";
this.description = `Performs BLAKE2s hashing on the input.
<br><br>BLAKE2s is a flavour of the BLAKE cryptographic hash function that is optimized for 8- to 32-bit platforms and produces digests of any size between 1 and 32 bytes.
<br><br>Supports the use of an optional key.`;
this.infoURL = "https://wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE2";
this.inputType = "ArrayBuffer";
this.outputType = "string";
this.args = [
{
"name": "Size",
"type": "option",
"value": ["256", "160", "128"]
}, {
"name": "Output Encoding",
"type": "option",
"value": ["Hex", "Base64", "Raw"]
},
{
"name": "Key",
"type": "toggleString",
"value": "",
"toggleValues": ["UTF8", "Decimal", "Base64", "Hex", "Latin1"]
}
];
}
/**
* @param {ArrayBuffer} input
* @param {Object[]} args
* @returns {string} The input having been hashed with BLAKE2s in the encoding format speicifed.
*/
run(input, args) {
const [outSize, outFormat] = args;
let key = Utils.convertToByteArray(args[2].string || "", args[2].option);
if (key.length === 0) {
key = null;
} else if (key.length > 32) {
throw new OperationError(["Key cannot be greater than 32 bytes", "It is currently " + key.length + " bytes."].join("\n"));
}
input = new Uint8Array(input);
switch (outFormat) {
case "Hex":
return blakejs.blake2sHex(input, key, outSize / 8);
case "Base64":
return toBase64(blakejs.blake2s(input, key, outSize / 8));
case "Raw":
return Utils.arrayBufferToStr(blakejs.blake2s(input, key, outSize / 8).buffer);
default:
return new OperationError("Unsupported Output Type");
}
}
}
export default BLAKE2s;

8
src/core/operations/BSONDeserialise.mjs
View file

@ -4,9 +4,9 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import bsonjs from "bson";
import OperationError from "../errors/OperationError";
import Operation from "../Operation.mjs";
import bson from "bson";
import OperationError from "../errors/OperationError.mjs";
/**
* BSON deserialise operation
@ -36,8 +36,6 @@ class BSONDeserialise extends Operation {
run(input, args) {
if (!input.byteLength) return "";
const bson = new bsonjs();
try {
const data = bson.deserialize(new Buffer(input));
return JSON.stringify(data, null, 2);

8
src/core/operations/BSONSerialise.mjs
View file

@ -4,9 +4,9 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import bsonjs from "bson";
import OperationError from "../errors/OperationError";
import Operation from "../Operation.mjs";
import bson from "bson";
import OperationError from "../errors/OperationError.mjs";
/**
* BSON serialise operation
@ -36,8 +36,6 @@ class BSONSerialise extends Operation {
run(input, args) {
if (!input) return new ArrayBuffer();
const bson = new bsonjs();
try {
const data = JSON.parse(input);
return bson.serialize(data).buffer;

107
src/core/operations/BaconCipherDecode.mjs Normal file
View file

@ -0,0 +1,107 @@
/**
* @author Karsten Silkenbäumer [github.com/kassi]
* @copyright Karsten Silkenbäumer 2019
* @license Apache-2.0
*/
import Operation from "../Operation";
import {
BACON_ALPHABETS,
BACON_TRANSLATION_CASE, BACON_TRANSLATION_AMNZ, BACON_TRANSLATIONS, BACON_CLEARER_MAP, BACON_NORMALIZE_MAP,
swapZeroAndOne
} from "../lib/Bacon";
/**
* Bacon Cipher Decode operation
*/
class BaconCipherDecode extends Operation {
/**
* BaconCipherDecode constructor
*/
constructor() {
super();
this.name = "Bacon Cipher Decode";
this.module = "Default";
this.description = "Bacon's cipher or the Baconian cipher is a method of steganography devised by Francis Bacon in 1605. A message is concealed in the presentation of text, rather than its content.";
this.infoURL = "https://wikipedia.org/wiki/Bacon%27s_cipher";
this.inputType = "string";
this.outputType = "string";
this.args = [
{
"name": "Alphabet",
"type": "option",
"value": Object.keys(BACON_ALPHABETS)
},
{
"name": "Translation",
"type": "option",
"value": BACON_TRANSLATIONS
},
{
"name": "Invert Translation",
"type": "boolean",
"value": false
}
];
}
/**
* @param {string} input
* @param {Object[]} args
* @returns {string}
*/
run(input, args) {
const [alphabet, translation, invert] = args;
const alphabetObject = BACON_ALPHABETS[alphabet];
// remove invalid characters
input = input.replace(BACON_CLEARER_MAP[translation], "");
// normalize to unique alphabet
if (BACON_NORMALIZE_MAP[translation] !== undefined) {
input = input.replace(/./g, function (c) {
return BACON_NORMALIZE_MAP[translation][c];
});
} else if (translation === BACON_TRANSLATION_CASE) {
const codeA = "A".charCodeAt(0);
const codeZ = "Z".charCodeAt(0);
input = input.replace(/./g, function (c) {
const code = c.charCodeAt(0);
if (code >= codeA && code <= codeZ) {
return "1";
} else {
return "0";
}
});
} else if (translation === BACON_TRANSLATION_AMNZ) {
const words = input.split(/\s+/);
const letters = words.map(function (e) {
if (e) {
const code = e[0].toUpperCase().charCodeAt(0);
return code >= "N".charCodeAt(0) ? "1" : "0";
} else {
return "";
}
});
input = letters.join("");
}
if (invert) {
input = swapZeroAndOne(input);
}
// group into 5
const inputArray = input.match(/(.{5})/g) || [];
let output = "";
for (let i = 0; i < inputArray.length; i++) {
const code = inputArray[i];
const number = parseInt(code, 2);
output += number < alphabetObject.alphabet.length ? alphabetObject.alphabet[number] : "?";
}
return output;
}
}
export default BaconCipherDecode;

101
src/core/operations/BaconCipherEncode.mjs Normal file
View file

@ -0,0 +1,101 @@
/**
* @author Karsten Silkenbäumer [github.com/kassi]
* @copyright Karsten Silkenbäumer 2019
* @license Apache-2.0
*/
import Operation from "../Operation";
import {
BACON_ALPHABETS,
BACON_TRANSLATIONS_FOR_ENCODING, BACON_TRANSLATION_AB,
swapZeroAndOne
} from "../lib/Bacon";
/**
* Bacon Cipher Encode operation
*/
class BaconCipherEncode extends Operation {
/**
* BaconCipherEncode constructor
*/
constructor() {
super();
this.name = "Bacon Cipher Encode";
this.module = "Default";
this.description = "Bacon's cipher or the Baconian cipher is a method of steganography devised by Francis Bacon in 1605. A message is concealed in the presentation of text, rather than its content.";
this.infoURL = "https://wikipedia.org/wiki/Bacon%27s_cipher";
this.inputType = "string";
this.outputType = "string";
this.args = [
{
"name": "Alphabet",
"type": "option",
"value": Object.keys(BACON_ALPHABETS)
},
{
"name": "Translation",
"type": "option",
"value": BACON_TRANSLATIONS_FOR_ENCODING
},
{
"name": "Keep extra characters",
"type": "boolean",
"value": false
},
{
"name": "Invert Translation",
"type": "boolean",
"value": false
}
];
}
/**
* @param {string} input
* @param {Object[]} args
* @returns {string}
*/
run(input, args) {
const [alphabet, translation, keep, invert] = args;
const alphabetObject = BACON_ALPHABETS[alphabet];
const charCodeA = "A".charCodeAt(0);
const charCodeZ = "Z".charCodeAt(0);
let output = input.replace(/./g, function (c) {
const charCode = c.toUpperCase().charCodeAt(0);
if (charCode >= charCodeA && charCode <= charCodeZ) {
let code = charCode - charCodeA;
if (alphabetObject.codes !== undefined) {
code = alphabetObject.codes[code];
}
const bacon = ("00000" + code.toString(2)).substr(-5, 5);
return bacon;
} else {
return c;
}
});
if (invert) {
output = swapZeroAndOne(output);
}
if (!keep) {
output = output.replace(/[^01]/g, "");
const outputArray = output.match(/(.{5})/g) || [];
output = outputArray.join(" ");
}
if (translation === BACON_TRANSLATION_AB) {
output = output.replace(/[01]/g, function (c) {
return {
"0": "A",
"1": "B"
}[c];
});
}
return output;
}
}
export default BaconCipherEncode;

7
src/core/operations/Bcrypt.mjs
View file

@ -4,8 +4,9 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Operation from "../Operation.mjs";
import bcrypt from "bcryptjs";
import { isWorkerEnvironment } from "../Utils.mjs";
/**
* Bcrypt operation
@ -19,7 +20,7 @@ class Bcrypt extends Operation {
super();
this.name = "Bcrypt";
this.module = "Hashing";
this.module = "Crypto";
this.description = "bcrypt is a password hashing function designed by Niels Provos and David Mazi\xe8res, based on the Blowfish cipher, and presented at USENIX in 1999. Besides incorporating a salt to protect against rainbow table attacks, bcrypt is an adaptive function: over time, the iteration count (rounds) can be increased to make it slower, so it remains resistant to brute-force search attacks even with increasing computation power.<br><br>Enter the password in the input to generate its hash.";
this.infoURL = "https://wikipedia.org/wiki/Bcrypt";
this.inputType = "string";
@ -44,7 +45,7 @@ class Bcrypt extends Operation {
return await bcrypt.hash(input, salt, null, p => {
// Progress callback
if (ENVIRONMENT_IS_WORKER())
if (isWorkerEnvironment())
self.sendStatusMessage(`Progress: ${(p * 100).toFixed(0)}%`);
});

8
src/core/operations/BcryptCompare.mjs
View file

@ -4,8 +4,10 @@
* @license Apache-2.0
*/
import Operation from "../Operation";
import Operation from "../Operation.mjs";
import bcrypt from "bcryptjs";
import { isWorkerEnvironment } from "../Utils.mjs";
/**
* Bcrypt compare operation
@ -19,7 +21,7 @@ class BcryptCompare extends Operation {
super();
this.name = "Bcrypt compare";
this.module = "Hashing";
this.module = "Crypto";
this.description = "Tests whether the input matches the given bcrypt hash. To test multiple possible passwords, use the 'Fork' operation.";
this.infoURL = "https://wikipedia.org/wiki/Bcrypt";
this.inputType = "string";
@ -43,7 +45,7 @@ class BcryptCompare extends Operation {
const match = await bcrypt.compare(input, hash, null, p => {
// Progress callback
if (ENVIRONMENT_IS_WORKER())
if (isWorkerEnvironment())
self.sendStatusMessage(`Progress: ${(p * 100).toFixed(0)}%`);
});

Some files were not shown because too many files have changed in this diff Show more