I’m currently troubleshooting an issue with my Kubernetes clusters where pods keep getting evicted, and this is happening across namespaces as well.
The issue now that I am faced with, is being able to keep ontop of the issues. When I run:
kubectl get pods -A | grep Evicted
I’m presented with 100’s of returned results.
So to quickly clean this up, I can run the following command:
kubectl get pods -A | grep Evicted | awk '{print $1,$2,$4}' | xargs kubectl delete pod $2 -n $1
Breaking down the command:
- Get all pods across all namespaces
- Filter by term “Evicted”
- Manipulate the output by selecting the data in field 1, 2 and 4
- Use xargs to read from the standard output to place the data from the previous pipe into the “kubectl” command.
This command will cycle and remove everything you need. You can use this command line for other status of pods if needed.
Now I need to get back to troubleshooting my cluster issues.
Regards
To slightly improve, check for evicted pods with awk directly, instead of preprocessing with grep.
`kubectl get pods -A | awk ‘/Evicted/{print $1,$2}’ | xargs kubectl delete pod $2 -n $1`
Thank you for this Rick!
The xargs syntax is not valid.
The correct one is:
“`
kubectl get pods -A | awk ‘/Evicted/{print $1,$2}’ | xargs -r kubectl delete pod -n
“`
The original line works because $1 and $2 are replaced by the shell with empty strings, then the values returned by the awk output are appended at the end, resulting in a line equivalent to the one above.
The `-r` addition is to avoid an error in case there are no evicted pods in the cluster.
Thank you for this Marco, I can add this to my TIL list 🙂