What does it mean to keep the Covenant?

It means to accept HaShem as your King and obey Him.

If you have not been obedient to HaShem, then you need to repent, which means to return to the Torah, the Covenant, which God made with everyone, as it is written:

Deuteronomy 29:12-15
12 You are standing here in order to enter into a covenant with HaShem your God, a covenant HaShem is making with you this day and sealing with an oath,
13 to confirm you this day as his people, that he may be your God as he promised you and as he swore to your fathers, Abraham, Isaac and Jacob.
14 I am making this covenant, with its oath, not only with you
15 who are standing here with us today in the presence of HaShem our God but also with those who are not here today.

Everyone currently not keeping the Covenant, is someone who willfully and rebelliously disobeys HaShem and the One in Whom His Name is in, and by such rebellion are blotted out of HaShem's book and thus will not inherit a place in the World to Come, as it is written:

Exodus 23:21
Pay attention to him and listen to what he says. Do not rebel against him; he will not forgive your rebellion, since my Name is in him.

Exodus 32:33
HaShem replied to Moses, "Whoever has sinned against me I will blot out of my book.

HaShem says clearly those who sin against him will be blotted out of his book. So if we rebel against the one who has HaShem's name, he does not forgive us, we have sinned against HaShem, and are blotted out of his book.

Who then, is the one who has HaShem's name in him? It is written:

Exodus 32:20-22
20 See, I am sending a messenger ahead of you to guard you along the way and to bring you to the place I have prepared.

He is the one who is sent from HaShem to guard us on the way, to bring us to the place (the Beit HaMikdash, Temple Mount, Jerusalem, Israel). He, like Moses, is the Messiah who will lead us out from the desert into Israel, and it is he who guards us on the way by teaching us the Torah, and who brings us to the place, which HaShem has prepared. As it is written:

Deuteronomy 18:15
The LORD your God will raise up for you a prophet like me from among your own brothers. You must listen to him.

Deuteronomy 18:18-19
18 I will raise up for them a prophet like you from among their brothers; I will put my words in his mouth, and he will tell them everything I command him.
19 If anyone does not listen to my words that the prophet speaks in my name, I myself will call him to account.

This then is how we keep the Covenant: obey the prophet, the one like Moses, in whom the Name of HaShem is in, and is the one who tells us everything God commands.

Who then is this that we might obey and therefore be kept from having our names blotted out of HaShem's book, and not be called too account if we follow him? King Messiah.

There is only one candidate in all history that meets the qualifications for King Messiah: Yeshua ben Yosef shel Netzaret. It is Him we must obey, for like Moses, He teaches the Torah of Moses, and tells us everything God commands. As is it testified concerning him:

John 1:45
Philip found Nathanael and told him, "We have found the one Moses wrote about in the Law, and about whom the prophets also wrote—Jesus of Nazareth, the son of Joseph."

Acts 3:22-26
22 For Moses said, 'The Lord your God will raise up for you a prophet like me from among your own people; you must listen to everything he tells you.
23 Anyone who does not listen to him will be completely cut off from among his people.'
24"Indeed, all the prophets from Samuel on, as many as have spoken, have foretold these days.
25 And you are heirs of the prophets and of the covenant God made with your fathers. He said to Abraham, 'Through your offspring all peoples on earth will be blessed.
26 When God raised up his servant, he sent him first to you to bless you by turning each of you from your wicked ways."

Acts 7:37
This is that Moses who told the Israelites, 'God will send you a prophet like me from your own people.'

As testified, Yeshua is that prophet, by those who lived with him and saw everything that he did.
He is that messenger who guards us on the Way to the World to Come.
He is the one in who HaShem's name is in and is who we must obey.

Deuteronomy 6:4-5
Hear, O Israel: HaShem our God, HaShem is one. Love HaShem your God with all your heart and with all your soul and with all your strength.

HaShem is our King, Savior, and Redeemer. We have no other than him.

It is only when we submit to the Word of HaShem, King Messiah Yeshua, do we see what it truly means to keep the Covenant. For then we realize that our keeping (or lack thereof) is our claim of who is our King and what he does - since he has commanded us his Torah, the Torah that was given to Moses, and expounded upon by the Prophets, Sages, the Men of the Great Assembly, the talmidim of Messiah Yeshua, and the talmidim of the Great Assembly, and their students throughout the millennia. It is our obedience to the Torah that certifies who we are - it proves that we are the King's subjects, and it is by obedience to the Torah that we will be rewarded for our faithfulness, in the World Come.

Furthermore, one can not obey the King if they do not love God, as Messiah Yeshua clearly taught that it was the greatest commandment. He said that "all the Torah hangs" from "love HaShem your God" and "love your neighbor as your self," which means there is nothing in the Torah that is outside the category of loving God and others. This understanding is also understood by the Sages.

All the Torah is instructive in the pursuit of loving of God and loving one's neighbor, and to that end, we must also realize that we can not keep Torah if we do not love God. There is nothing in the Torah that we should not do if given the opportunity to keep. If we love something or someone else in the place of the one true God, then we are engaging in idolatry, and by definition are hating the one true God - and this is what happens anytime we transgress the Torah.

So then, rejecting idolatry, returning to the Torah and obeying it, and by it submitting to the lordship of Messiah Yeshua, and keeping the Torah out of love for God and others, is how one keeps the Covenant, and is the first step toward true conversion to Judaism. All else follows.

Questions

Does keeping the Covenant include getting circumcised?
Some ask if "keeping the Covenant" includes keeping the Torah's teaching on circumcision for males. It most certainly it does! If it's in the Torah, then the Messiah taught it! He can only teach God's commands, and this includes obedience to Him and His ways as instructed in the entire Torah, down to every jot and tittle.

See: is circumcision required for keeping the Covenant?

Does keeping the Covenant mean converting to Judaism?
Others ask if circumcision is required, is then conversion to Judaism required as part of keeping the Covenant? The answer is: Yes it is. As a Covenant member, and a disciple of Yeshua, your ultimate obligation is to become his disciple and follow Him. This means obeying the Torah, and by obeying the Torah you are required to convert to Judaism in order to fulfill the mitzvot (commandments) the Master taught his disciples to follow (which is the Torah of Moses), which includes observing the Passover "in remembrance" of Him, and which the Torah teaches of the Passover: "no alien may eat of it." (Exodus 12:43)

See Giyur (Conversion)