MockHttpSocket and MockHttpAgent are not being garbage collected
#2405
Labels
bug
Something isn't working
help wanted
Extra attention is needed
needs:triage
Issues that have not been investigated yet.
scope:node
Related to MSW running in Node
Comments
rossipedia
added
bug
|
This is causing a large memory leak meaning we have to restart our production pods every day. Any update / acknowledgement of the issue would be great. |
|
@kettanaito sorry for the direct ping and we really appreciate your work here, but this is causing us at Docker significant issues as use MSW in production to host a demo environment that matches our e2e tests and we have to restart production pods constantly due to this MSW memory leak. We can potentially contribute engineering effort if you can help point us in the right direction as it would be a large lift to migrate away from MSW for this. |
|
@mikeparker, I am sorry to hear that this is blocking you. If you have time, could you please see what in My initial guess is that it can be request or response parser, although we try to free it at the first available moment (1, 2). I will try to look into this but I am overburdened as of now. I would be grateful if you landed me a hand investigating this so I can roll out the appropriate fix as soon as we know what's causing the issue. Thanks. |
|
Another potential problem area is that the agent/socket interface may not have some of the methods implemented that result in it not being disposed of correctly. This can be proven/disproven by constructing and observing an actual |
|
I think I'm seeing something related: It would be nice to be able to use msw without the websocket interceptors being applied. |
|
@85B622ACA6, I don't believe we interfere with request at that level to result in the unexpected |
kettanaito
changed the title
MockHttpSocket and MockHttpAgent are not being garbage collected
|
I believe you are correct. After playing around with it some more, the results I was seeing seem to be related to how msw interacts with the 'ws' WebSocket client. I hate to go of topic, but is msw expected to play nicely with clients from the 'ws' package? |
|
I've opened a pull request to try to measure the memory footprint of the interceptors in mswjs/interceptors#702. While I cannot see quite the same memory heap given more-or-less the same reproduction scenario, the memory snapshot with interceptors is ~50MB larger for 10,000 requests. Alas, I have no idea how to navigate the heap snapshot to understand why |
Yes, I believe so. We are using |
|
For node versions <22 too? (22 introduced the built-in WebSocket client) |
|
@85B622ACA6, yes, as long as |
That's a great idea, lemme poke around at that and see what pops up |
|
Ok so playing around with import net from 'node:net';
const handler = {
get(t, p, r) {
const v = Reflect.get(t, p, r);
console.log(`-----------------------------------------------------`);
console.log(
`get socket[${p.toString()}]:\n${
typeof v === 'function' ? 'function' : v
}`
);
if (typeof v === 'function') {
// trap for arguments
return new Proxy(v, {
apply(target, thisArg, args) {
console.log(`-----------------------------------------------------`);
console.log(`-> calling socket[${p.toString()}](${args.join(', ')})`);
return Reflect.apply(target, thisArg, args);
},
});
}
return v;
},
};
const server = net.createServer((c) => {
const s = new Proxy(c, handler);
s.pipe(s);
});
server.on('error', (err) => {
throw err;
});
server.listen(8124, () => {
console.log('server bound');
});Kicking that up and then sending a 'foo' string into it with: $ echo "foo" | nc localhost 8124Gives the following output. I'll still dig in some more, but maybe you can see something I don't 😅 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Prerequisites
Environment check
mswversionNode.js version
22.12.0
Reproduction repository
https://github.com/rossipedia/msw-leak-test
Reproduction steps
Instructions are in the reproduction repo, but they are essentially:
npm install./demo.shCurrent behavior
Note
This is a follow up to my colleague's issue #2390
Enabling MSW with no handlers and then load-testing using
abcauses heap usage to balloon, even when manually invoking garbage collection.As best I can tell, the largest percentage of retained objects are primarily:
MockHttpSocketMockHttpAgentI do not believe this is related to
express, as onlyClientRequestis intercepted, and I don't thinkexpressorhttp.createServeruseClientRequest, onlyIncomingMessageandServerResponse(I could be wrong about that though).Expected behavior
Memory usage should be stable under load when simply enabling MSW in a node application.
The text was updated successfully, but these errors were encountered: