952

I just recently upgraded to npm@5. I now have a package-lock.json file with everything from package.json. I would expect that, when I run npm install that the dependency versions would be pulled from the lock file to determine what should be installed in my node_modules directory. What's strange is that it actually ends up modifying and rewriting my package-lock.json file.

For example, the lock file had typescript specified to be at version 2.1.6. Then, after the npm install command, the version was changed to 2.4.1. That seems to defeat the whole purpose of a lock file.

What am I missing? How do I get npm to actually respect my lock file?

6
  • 9
    The same problem but using yarn github.com/yarnpkg/yarn/issues/570 (very instructive)
    – Yves M.
    Commented Jul 11, 2017 at 14:02
  • 5
    I am having the same issue. My package-lock.json gets regenerated when i run npm install. This smells like a npm bug. Do you use your own registry?
    – HaNdTriX
    Commented Jul 13, 2017 at 13:34
  • 3
    See also npm5 equivalent to yarn's --pure-lockfile flag?
    – Yves M.
    Commented Jul 22, 2017 at 21:20
  • 2
    @YvesM. --no-save prevents changing the lockfile, but it doesn't affect the goofy first-level dependency upgrading that the OP mentions.
    – Ross Allen
    Commented Aug 4, 2017 at 17:16
  • This seems to also happen with npm6 - I ran npm i without changing anything, and my package-lock.json was modified (the versions under all the packages in requires changed). It seems intended and not to break anything? More info here Commented May 1, 2018 at 15:19

12 Answers 12

636

Update 3: As other answers point out as well, the npm ci command got introduced in npm 5.7.0 as additional way to achieve fast and reproducible builds in the CI context. See the documentation and npm blog for further information.


Update 2: The issue to update and clarify the documentation is GitHub issue #18103.


Update 1: The behaviour that was described below got fixed in npm 5.4.2: the currently intended behaviour is outlined in GitHub issue #17979.


Original answer (pre-5.4.2): The behaviour of package-lock.json was changed in npm 5.1.0 as discussed in issue #16866. The behaviour that you observe is apparently intended by npm as of version 5.1.0.

That means that package.json can override package-lock.json whenever a newer version is found for a dependency in package.json. If you want to pin your dependencies effectively, you now must specify the versions without a prefix, e.g., you need to write them as 1.2.0 instead of ~1.2.0 or ^1.2.0. Then the combination of package.json and package-lock.json will yield reproducible builds. To be clear: package-lock.json alone no longer locks the root level dependencies!

Whether this design decision was good or not is arguable, there is an ongoing discussion resulting from this confusion on GitHub in issue #17979. (In my eyes it is a questionable decision; at least the name lock doesn't hold true any longer.)

One more side note: there is also a restriction for registries that don’t support immutable packages, such as when you pull packages directly from GitHub instead of npmjs.org. See this documentation of package locks for further explanation.

21
  • 83
    What the hack is npm update for then ? :o I've had same feeling that npm install updated deps, but I doesn't want to believe it.. but seems like it's sadly true.. Anyway there is still option to use npm shrinkwrap to lock deps, but definitely name package-lock is incorrect as it does not freeze, nor lock dependencies..
    – Jurosh
    Commented Sep 7, 2017 at 6:48
  • 383
    What a mess! The worlds largest package manager yet it doesn't have documentation on how it should work. Everyone is guessing about what it should do and it turns into a war of opinions. Discussion is good but should happen prior to a release into the wild. At some point someone needs to make the final call and then in can get implemented, documented and released. PHP was designed by committee and ad-hoc'd together and look how it turned out. I'd hate to see the same thing happen to a tool this critical and widely used. Commented Sep 15, 2017 at 7:45
  • 124
    Then, what is the point of using package-lock ? I thought it would create the same environment in different workspaces but turns out it is doing nothing
    – laltin
    Commented Oct 30, 2017 at 20:32
  • 31
    "Then the combination of package.json and package-lock.json will yield reproducible builds." What role does "package-lock.json" have here? Doesn't "package.json" alone already yield reproducible builds if no version prefixes are used? Commented Feb 27, 2018 at 11:46
  • 18
    @JānisElmeris I think package.json can't lock down deep dependencies... Commented Dec 11, 2018 at 20:30
249

I've found that there will be a new version of npm 5.7.1 with the new command npm ci, that will install from package-lock.json only

The new npm ci command installs from your lock-file ONLY. If your package.json and your lock-file are out of sync then it will report an error.

It works by throwing away your node_modules and recreating it from scratch.

Beyond guaranteeing you that you'll only get what is in your lock-file it's also much faster (2x-10x!) than npm install when you don't start with a node_modules.

As you may take from the name, we expect it to be a big boon to continuous integration environments. We also expect that folks who do production deploys from git tags will see major gains.

11
  • 217
    This should be the default behavior if a lockfile exists. Commented Jul 12, 2018 at 16:48
  • 19
    So they changed how npm i works, only to bring it back in as npm ci months later? Commented Jul 13, 2018 at 0:32
  • 6
    I'm still confused. The documentations says "Make sure you have a package-lock and an up-to-date install: npm install" before running the command npm ci in that project. Doesn't npm install overwrite the package-lock.json file?
    – adiga
    Commented Jun 22, 2019 at 10:41
  • 4
    AFAIK: @adiga - starting with version 5.4, npm only changes the lock file if necessary to do so, to meet the spec in packages.json. So if packages used to say thatpackage: 1, and lock says ..: 1.0.4, dev can edit to say thatpackage: 2 - and that will force lock file to change, because 1.0.4 isn't compatible with the newly specified range. If don't change packages.json, will stay locked at exact version, until delete lock file. [If doesn't stay locked, and didn't change packages.json, file a bug report.] Commented Oct 30, 2019 at 20:56
  • 2
    Spent my entire day on it. I spent my entire day on this fundamental issue :( :(
    – Omar Tariq
    Commented Nov 7, 2019 at 15:52
186

Short Answer:

  • npm install honors package-lock.json only if it satisfies the requirements of package.json.
  • If it doesn't satisfy those requirements, packages are updated & package-lock is overwritten.
  • If you want the install to fail instead of overwriting package-lock when this happens, use npm ci.

Here is a scenario that might explain things (Verified with NPM 6.3.0)

You declare a dependency in package.json like:

"depA": "^1.0.0"

Then you do, npm install which will generate a package-lock.json with:

"depA": "1.0.0"

Few days later, a newer minor version of "depA" is released, say "1.1.0", then the following holds true:

npm ci       # respects only package-lock.json and installs 1.0.0

npm install  # also, respects the package-lock version and keeps 1.0.0 installed 
             # (i.e. when package-lock.json exists, it overrules package.json)

Next, you manually update your package.json to:

"depA": "^1.1.0"

Then rerun:

npm ci      # will try to honor package-lock which says 1.0.0
            # but that does not satisfy package.json requirement of "^1.1.0" 
            # so it would throw an error 

npm install # installs "1.1.0" (as required by the updated package.json)
            # also rewrites package-lock.json version to "1.1.0"
            # (i.e. when package.json is modified, it overrules the package-lock.json)
13
  • 10
    This is indeed the intended behavior of a "lock" file. Apparently, it was not the case with older versions of NPM.
    – Blockost
    Commented Jan 15, 2019 at 21:57
  • 1
    Then how does npm track the last update to package.json? What happens when you move your package.json and package-lock.json to another computer? How does npm in new computer know whether package.lock is the original one or it has been updated, to decide whether it needs to update package-lock.json or not? Commented Jun 8, 2019 at 8:15
  • 9
    @LahiruChandima It does not really track updates. npm install will use the locked versions from package-lock.json unless it does not satisfy the package.json in which case it installs package.json and rebuilds package-lock.json accordingly. If you changed your package.json in such a way that existing package-lock still satisfies the updated package.json it will continue to use that package-lock Commented Jun 28, 2019 at 13:32
  • 1
    If you already have a module in node_modules that meets the requirements of package.json, then npm install does nothing, regardless of package-lock.json. We have to explicitly update packages even when there are updates available that match the semver specified in package.json. At least that has been my experience for years. Commented Sep 12, 2019 at 18:29
  • 1
    @ToolmakerSteve I was also skeptical of the behavior @carlin.scott reported, but I just tested it, and in fact he's correct. If the version within node_modules satisfies the range in package.json, and there is no package-lock.json file, npm will not update the module when running npm install. I guess it's fine since you can use npm update (or npm-check for latest) to update dependencies, and this behavior is faster for the case of someone just adding one entry to package.json, and not wanting unrelated packages to update themselves to the latest that satisfies the sem-ver range.
    – Venryx
    Commented Nov 13, 2019 at 10:33
122

Use the newly introduced

npm ci

npm ci promises the most benefit to large teams. Giving developers the ability to “sign off” on a package lock promotes more efficient collaboration across large teams, and the ability to install exactly what is in a lockfile has the potential to save tens if not hundreds of developer hours a month, freeing teams up to spend more time building and shipping amazing things.

Introducing npm ci for faster, more reliable builds

8
  • 5
    this seems to be correct to me? can anyone else confirm?
    – phouse512
    Commented Jun 28, 2018 at 20:22
  • 9
    @phouse512 This is correct. We pretty much only use npm ci, and only use npm install if updating or installing new packages. Commented Aug 6, 2018 at 9:08
  • 1
    Recent comments, etc. This is the answer I'm going with. Too bad they coudln't fix the horrible snafu, but if the new gospel is "npm ci", then fine. I can adapt.
    – Svend
    Commented Aug 14, 2018 at 11:12
  • 2
    Too bad it always deletes an existing node_modules directory and rebuilds locally, even if that is an otherwise empty but important symlink. :( Commented Jun 13, 2019 at 0:57
  • 2
    @ToolmakerSteve Don't hold your breath! I think deleting the contents of a directory would be magnitudes slower than just deleting the directory. You would have to enumerate the contents then issue a series of delete commands rather than just the one delete command to the O/S. With the performance issues previously levelled at npm and the improvement using npm ci I expect they would be very reluctant to introduce anything that could reduce performance for a fairly uncommon use case. You might want to check out pnpm.js.org though that makes use of hard links to reduce disk usage.
    – Caltor
    Commented Dec 19, 2019 at 10:25
42

Use the npm ci command instead of npm install.

"ci" stands for "clean install".

It will install the project dependencies based on the package-lock.json file instead of the lenient package.json file dependencies.

It will produce identical builds to your team mates and it is also much faster.

You can read more about it in this blog post: https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable

12
  • 7
    ci refers to "continuous integration", as mentioned in the docs and blog post announcing the command: blog.npmjs.org/post/171556855892/… Commented Jun 13, 2019 at 1:04
  • 4
    "And it is also much faster" - it will delete node_modules folder and re-create it from scratch. Is it really much faster? Does npm install delete node_modules folder, too?
    – izogfif
    Commented Nov 27, 2019 at 11:29
  • 3
    I think the speed comes from npm not needing to calculate what packages to download. Think of it like npm install has to resolve all the package dependencies when run. npm ci is just a shopping list of "get these exact modules". Commented Nov 27, 2019 at 11:58
  • 4
    ci stands for clean install actually. Commented Sep 16, 2021 at 8:51
  • 1
    stands for "clean install", I just read a post by the person who built it. Can't find the link at the moment..
    – MattoMK
    Commented Feb 25, 2022 at 1:47
9

In the future, you will be able to use a --from-lock-file (or similar) flag to install only from the package-lock.json without modifying it.

This will be useful for CI, etc. environments where reproducible builds are important.

See https://github.com/npm/npm/issues/18286 for tracking of the feature.

2
  • I doubt it. How if dependencies are different for different operation systems, how can you force install something that would not work? Commented Jan 24, 2018 at 23:27
  • 7
    @YevgeniyAfanasyev Instead of that flag, it was implemented as npm ci which also handles your question.
    – spex
    Commented Mar 30, 2018 at 2:45
9

It appears this issue is fixed in npm v5.4.2

https://github.com/npm/npm/issues/17979

(Scroll down to the last comment in the thread)

Update

Actually fixed in 5.6.0. There was a cross platform bug in 5.4.2 that was causing the issue to still occur.

https://github.com/npm/npm/issues/18712

Update 2

See my answer here: https://stackoverflow.com/a/53680257/1611058

npm ci is the command you should be using when installing existing projects now.

4
  • 7
    I'm using 5.4.2 and it's still resulting in the modification of my package-lock.json when npm i. For instance, the module fsevents is removed when I npm i on a machine that does not support fsevents and then the module is re-added when one npm i again on a machine that does.
    – hrdwdmrbl
    Commented Oct 3, 2017 at 22:12
  • Then you should raise a new issue in the npm GitHub repo explaining this. If it doesn't work how they say it is supposed to work then they see it as a high priority bug that urgently needs fixing. Commented Oct 3, 2017 at 22:23
  • @hrdwdmrbl I'm seeing the same fsevents drop in my package-lock.json with [email protected] while collaborating with Mac OS X contributors. If you haven't opened an issue, I will.
    – AL the X
    Commented Nov 16, 2017 at 20:48
  • @hrdwdmrbl I found that (and the long thread of associated issues) after I left my comment and forgot to come back to SO to update my comment. Thanks for getting my back. Everything is fine.
    – AL the X
    Commented Nov 22, 2017 at 3:23
8

Probably you should use something like this

npm ci

Instead of using npm install if you don't want to change the version of your package.

According to the official documentation, both npm install and npm ci install the dependencies which are needed for the project.

The main difference is, npm install does install the packages taking packge.json as a reference. Where in the case of npm ci, it does install the packages taking package-lock.json as a reference, making sure every time the exact package is installed.

7

You probably have something like:

"typescript":"~2.1.6"

in your package.json which npm updates to the latest minor version, in your case being 2.4.1

Edit: Question from OP

But that doesn't explain why "npm install" would change the lock file. Isn't the lock file meant to create a reproducible build? If so, regardless of the semver value, it should still use the same 2.1.6 version.

Answer:

This is intended to lock down your full dependency tree. Let's say typescript v2.4.1 requires widget ~v1.0.0. When you npm install it grabs widget v1.0.0. Later on your fellow developer (or CI build) does an npm install and gets typescript v2.4.1 but widget has been updated to widget v1.0.1. Now your node module are out of sync. This is what package-lock.json prevents.

Or more generally:

As an example, consider

package A:

{ "name": "A", "version": "0.1.0", "dependencies": { "B": "<0.1.0" } }

package B:

{ "name": "B", "version": "0.0.1", "dependencies": { "C": "<0.1.0" } }

and package C:

{ "name": "C", "version": "0.0.1" }

If these are the only versions of A, B, and C available in the registry, then a normal npm install A will install:

[email protected] -- [email protected] -- [email protected]

However, if [email protected] is published, then a fresh npm install A will install:

[email protected] -- [email protected] -- [email protected] assuming the new version did not modify B's dependencies. Of course, the new version of B could include a new version of C and any number of new dependencies. If such changes are undesirable, the author of A could specify a dependency on [email protected]. However, if A's author and B's author are not the same person, there's no way for A's author to say that he or she does not want to pull in newly published versions of C when B hasn't changed at all.


OP Question 2: So let me see if I understand correctly. What you're saying is that the lock file specifies the versions of the secondary dependencies, but still relies on the fuzzy matching of package.json to determine the top-level dependencies. Is that accurate?

Answer: No. package-lock locks the entire package tree, including the root packages described in package.json. If typescript is locked at 2.4.1 in your package-lock.json, it should remain that way until it is changed. And lets say tomorrow typescript releases version 2.4.2. If I checkout your branch and run npm install, npm will respect the lockfile and install 2.4.1.

More on package-lock.json:

package-lock.json is automatically generated for any operations where npm modifies either the node_modules tree, or package.json. It describes the exact tree that was generated, such that subsequent installs are able to generate identical trees, regardless of intermediate dependency updates.

This file is intended to be committed into source repositories, and serves various purposes:

Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.

Provide a facility for users to "time-travel" to previous states of node_modules without having to commit the directory itself.

To facilitate greater visibility of tree changes through readable source control diffs.

And optimize the installation process by allowing npm to skip repeated metadata resolutions for previously-installed packages.

https://docs.npmjs.com/files/package-lock.json

11
  • 38
    But that doesn't explain why "npm install" would change the lock file. Isn't the lock file meant to create a reproducible build? If so, regardless of the semver value, it should still use the same 2.1.6 version. Commented Jul 10, 2017 at 21:53
  • 4
    And that's the thing I'm saying. My package lock file says [email protected] but when I run npm install, the entry is replaced with [email protected]. Commented Jul 11, 2017 at 16:56
  • 6
    I've experienced this same issue. In our CI/CD, the package-lock.json gets pulled down and then we run npm install, but the package-lock.json file is modified and we have to perform a reset before we can pull the next changes. Commented Jul 24, 2017 at 19:42
  • 20
    I don't get it. How is this a "lock" file if subsequent installs might still do upgrades?!
    – Ross Allen
    Commented Aug 4, 2017 at 17:17
  • 5
    I think they started with the idea of having this file as "info" and "lock" and then, decided it will be only an "info" file. Better name would be "package-info.json". I would love to have a "npm install -lock" which will install from "package-lock.json" and ignore "package.json" Commented Sep 7, 2017 at 2:20
5

EDIT: the name "lock" is a tricky one, its NPM trying to catch up with Yarn. It isn't a locked file whatsoever. package.json is a user-fixed file, that once "installed" will generate node_modules folder tree and that tree will then be written in package-lock.json. So you see, its the other way around - dependency versions will be pulled from package.json as always, and package-lock.json should be called package-tree.json

(hope this made my answer clearer, after so many downvotes)


A simplistic answer: package.json have your dependencies as usual, while package-lock.json is "an exact, and more importantly reproducible node_modules tree" (taken from npm docs itself).

As for the tricky name, its NPM trying to catch up with Yarn.

2
  • 1
    Because if you run npm install, package-lock will be updated. Commented Feb 19, 2018 at 16:34
  • Other answers already mentioned npm ci. In another answer of mine is explained the relationship between npm install, npm ci, package.json, and package.lock.json
    – Ictus
    Commented Aug 21, 2022 at 14:04
0

this site also has information about package-lock changes at subsequent npm i

https://github.com/npm/cli/issues/4263

-1

There is an open issue for this on their github page: https://github.com/npm/npm/issues/18712

This issue is most severe when developers are using different operating systems.

1
  • The rewrites in package-lock are intended, the issue is not a consequence of this Commented Jul 20, 2018 at 2:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.