Parallelize ES5 & ES2015 builds

[smnbbrv]

🚀 Feature request

Having a possibility to serve ES2015 is awesome! But, the build time was increased nearly twice because now instead of one build we have two.

Please provide ways to deal with this.

Command (mark with an x)

- [ ] new
- [x] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Description

The build time has significantly increased because ES5 and ES2015 builds are sequential.

Is there some reasoning behind this? Would it be faster to run them in parallel (at least with some explicit settings)?

Describe the solution you'd like

1. Parallelize those builds by default (or with special CLI flag probably)
2. Add a special CLI flag / configuration in angular.json that allows to build es5 and es2015 separately on different runners. This would be cool for CI because the builds could run on different physical devices (at least for CI tools where the dist folder gets mounted, e.g. Drone CI). This could cause problems with sharing the same dist directory (that is cleared before build), so this needs to be somehow resolved. However, if the option 1 is in place, I guess, this could be omitted

Describe alternatives you've considered

Disabling es2015 build :(

[alan-agius4]

Hi, can you kindly give some stats, such as the previous and current builds times thanks.

[smnbbrv]

Sure. Should have added this in the first post.

These are the timings before upgrade (time npm run build, where build = ng build --prod):

And this is after:

At the end of the day: 70.55s -> 135.15s

[filipesilva]

There are some concerns about progress reporting but they are minor.

As far as the individual build steps go, they are mostly parallelizable. Almost all compilation steps inside webpack run in the single thread.

There are three very expensive steps that only happen in prod builds: build-optimizer, module concatenation, and minification. Of these, only minification uses parallel processing.

But parallelizing things that themselves try to parallelize does not compose well. At some point the minification steps of both builds would look at the total available CPUs and say "right, so let's use them all". The best case scenario there is that both just bottleneck and get slower.

That'd be the end of it if CPU was the only relevant shared resource. But there is another one: memory. And that one is less easy to manage. Memory management is Node is mostly static, you define a maximum and there it goes. In Node 12 it figures out the maximum at the start, but that's still static from there on.

The really bad case is that at some point there are 2 minification processes that are processing a really big chunk (like main) and one/both will hit the memory and crash. To prevent that we'd have to ensure no two builds try to minify at the same time, or that minification doesn't use parallelism.

But I'm not sure that'd be such a great win insofar as build time. I can't imagine it would reduce it to half. Parallelization rarely does.

As an aside, we should figure out why the build time in webpack isn't the full time. It really should be in your first screenshot.

[alialtun14]

I upgraded my project to Angular 8 version. While upgrading process, I haven't met any error. But the compile time increased 3 times without any error.

Compile Before Angular 8: 1382.536ms -> main.cf29a89468d732f3f363.js (main) 12.4 MB
Compile After Angular 8: 3147.254ms -> main.da4147c72278bc767ee4.js (main) 12.3 MB

To minimize to compile time, is it possible to disabled compilation process for old browser. If It is, how can I disabled?

[filipesilva]

@alialtun14 you can read about how to configure it in https://angular.io/guide/deployment#configuring-differential-loading. You can just have the ES5 or ES2015 builds, or both.

[sod]

if the cli allows to select the target from the commandline via flag (e.g. something like ng build --prod --target=es5, you could parallelize it via:

ng build --prod --target=es5 & ng build --prod --target=es2015

or with something like https://yarnpkg.com/en/package/concurrently, so the stdout is a bit more organized.

Our you wait for bazel build, I'm pretty sure that thing will gladly exhaust all your cpu core available.

[clydin]

@sod That wouldn't provide a functioning index.html file.

[alialtun14]

@sod I have tried to use ng build --prod --target=es2015
I have met a error like this. Unknown option: '--target'

[sdon2]

@alialtun14 Change target configuration in tsconfig.json located @ root folder of your project. You'll get Single / Differential Loading builds according to your tsconfig.json configuration.

[clydin]

Angular CLI 8.3 and above no longer perform two full builds to support differential loading. Only an ES module compatible build is now created and then down-leveled directly to ES5. This can greatly reduce the time taken to perform a build especially for larger applications.

[smnbbrv]

Thank you @clydin , it works, and is fast as expected.

However it is really important to mention the following.

For those who struggles with this issue #14980 (which I find really nasty and killing the whole idea of ng update) it is important to upgrade not only angular cli to the latest version but also the @angular-devkit/build-angular which looks like to be implementing the whole thing.

   ...
   "@angular-devkit/build-angular": "^0.803.4",
   "@angular/cli": "^8.3.4",
   ...

Without this it still would consist of two builds.

[devel112]

Bundles have now increased in size

[clydin]

Please check the on-disk size. The ES5 bundles are actually smaller for a newly generated application.

[dsg-cmcmurray]

Bundles have now increased in size

I've noticed the same, it started occurring in 0.803.0. I've tried every version >= 0.803.0 (including the latest next version) and it has drastically increased bundle sizing of main and polyfills specifically. I've created an issue on the @angular-devkit/build-angular project: @angular-devkit/build-angular bundle sizing. For the time being, I'm staying on @angular-devkit/build-angular @ 0.802.2 but I've updated all of the @angular packages to 9.0.0-next.8 and all is good with build sizes. I'd rather sacrifice build time over runtime performance any day of the week.

[clydin]

As mentioned above, please verify the actual on disk size. There was a display error that could cause inaccurate size reporting that has since been fixed on master and will be available in the next patch release.

[dsg-cmcmurray]

As mentioned above, please verify the actual on disk size. There was a display error that could cause inaccurate size reporting that has since been fixed on master and will be available in the next patch release.

Well, it would help if I actually read all of the comments...yes, you are correct...I have validated that the disk size is not what is being reported. I will close the linked issue that I created earlier. Thank you!

[dsg-cmcmurray]

@clydin Upon second look, it appears that the compressed size does vary. You can see both the Gzip and Brotli files which I generate at build time are in fact larger on disk, even though the uncompressed bytes are roughly the same. Any idea why the compression sizes would vary that much? That's roughly 50k down the wire which is certainly nothing to sneeze at...

@angular-devkit/build-angular @ 0.802.2

-rw-r--r--    1 USERX  CORPORATE\Domain Users    866240 Sep 27 07:38 main-es2015.0a61ed855b7aa283d58b.js
-rw-r--r--    1 USERX  CORPORATE\Domain Users    183874 Sep 27 07:38 main-es2015.0a61ed855b7aa283d58b.js.br
-rw-r--r--    1 USERX  CORPORATE\Domain Users    226615 Sep 27 07:38 main-es2015.0a61ed855b7aa283d58b.js.gz

@angular-devkit/build-angular @ 0.803.6

-rw-r--r--   1 USERX  CORPORATE\Domain Users    866146 Sep 27 11:03 main-es2015.bd4c070b4cfc530a8271.js
-rw-r--r--   1 USERX  CORPORATE\Domain Users    235659 Sep 27 11:03 main-es2015.bd4c070b4cfc530a8271.js.br
-rw-r--r--   1 USERX  CORPORATE\Domain Users    294698 Sep 27 11:03 main-es2015.bd4c070b4cfc530a8271.js.gz

[clydin]

The ES2015 builds shouldn't change due to the differential loading process switch. It still generates an ES2015 build using the same method as before. It is the ES5 build that is created differently. Can you open a new issue with your findings as well as more details as to the versions used in each setup. Also if you have some time can you try with just 0.803.0? Thank you.

[dsg-cmcmurray]

The ES2015 builds shouldn't change due to the differential loading process switch. It still generates an ES2015 build using the same method as before. It is the ES5 build that is created differently. Can you open a new issue with your findings as well as more details as to the versions used in each setup. Also if you have some time can you try with just 0.803.0? Thank you.

@clydin Which repo would you recommend me opening the issue, @angular/angular-cli or @angular-devkit/build-angular? If the latter, I can simply reopen the issue that I linked to in my above comment and add/update some additional details. I already noted the setup versions there, and I will update the file size screenshots to be reflective of the size-on-disk and not reported output size. I have tried with every version >= 0.803.0, it seems this is the version which I start seeing an increase in the compressed files.

One obvious difference between the two file sets I pasted above is that the file hash and size are different, even though the source is the same (only difference is the version of @Angular-devkit)...so something is definitely different in the compilation process for ES2015 because the file's contents prior to hashing produces a different value. We know hashing is deterministic, so if the post-compilation output was the same for ES2015 between the versions then the file hash would be the same. This is not a surprise that hash and size vary, as build optimizations are constantly being made and I don't have any issue with this...but what I cannot wrap my head around just yet is why one has a higher compression ratio than the other.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}