Kim, who's 28, escaped North Korea in 2006 when he was 16, spent a year in China, then decided to find refuge in the U.S.
He’s one of several defectors ABC News spoke with to get their unique views on what they think of the summit.
Kim says he's "hopeful but cautiously skeptical," and thinks President Donald Trump should keep sanctions against North Korea in place to make sure Kim Jong Un doesn't acquire enough economic means to recharge the country's nuclear program.
"The U.S. priority is denuclearization and even to this day we don't know if North Korea and the U.S. share the same definition of what that is," said Kim, explaining why he had mixed feelings why he first heard about the possible meeting between the two leaders.
But his biggest concern is also personal: human rights.
"I hope that the regime would start, at least gradually, to make some commitment to improve human rights issue such as termination of the concentration camp, torture issues," said Kim. "Because that means the friends that I have left and friends that have grown up [with] in North Korea would potentially have a better chance of living."
"There are links between human rights abuse and the military program," said Lindsay Lloyd, the deputy director of Human Freedom at the Bush Institute. "It's one of the ways North Korea earns currency they use to acquire technology and acquire weapons and so forth."
"They have large programs of slave labor where they send North Koreans overseas to work in places like in Russia and China but also in the Middle East. The money doesn't go into those laborer's pockets, they all go to the government in Pyongyang and because of all the economic sanctions, North Korea is desperate to have access to western currency," Lloyd said, explaining it's important not to "lose sight of the condition of the North Korean people."
More than 31,000 North Korean defectors have entered South Korea, according to South Korea's Ministry of Unification. A relatively small number, about 220, came to the U.S., according to the Bush Institute.
While North Korean defectors are provided with a resettlement program catering to their specific needs in South Korea, a program that experts call "generous" and "outstanding" compared to the program in the U.S., defectors in the U.S. are treated like any other refugees.
Whether the defectors chose South Korea or the U.S. as their new home, the skepticism towards the leader of their previous home is shared.
"Without a clear condition stated from the North, Trump and Kim should not meet," said Sang Hak Park, a defector who has been living in Seoul for 18 years now. "North Korea has been lying for the past 27 years to the international society."
Kim Jong Il, Kim Jong Un's late father, broke nuclear agreements with the Clinton administration in the 1990s and the Bush administration a decade later.
"Kim Jong Un is following the exact same strategy that his father did. People are forgetting history," said Suzanne Scholte, the Chair of the North Korea Freedom Coalition, referring to the end of the 1990s and early 2000s when the North Korean regime was under "enormous stress."
"Kim Jong Il decided to have talks with South Korea, the 'sunshine policy' started, they were able to make all these promises… and what happened? Millions of people continue to die, suffer under this regime and they developed the nuclear program because they didn't follow through on the commitments," said Scholte.
"When they've lied eight times, why shouldn't they lie the ninth time?" said Park.
"What's even more important than nukes is the human rights issue," said Park. "It's the matter of whether our parents, brothers and sisters are living another 70 years as slaves of the leader or becoming free. It's an incredibly crucial time for us."
Park's family defected to Seoul in 1999. Park, who says he lived an upper-class life in Pyongyang, says he was lucky that his father was a former North Korean spy who disguised himself as a Japanese businessman in Seoul. Because of that his family had access to government officials helping them escape the country relatively easily.
"I want to ask Mr. Trump, please be on the side of the North Korean defectors and just say two things at the summit: denuclearization and termination of political prison camps. If he just says those two things, he will be remembered as a great president in history," said Park.
Although defectors regret the lack of human rights issues mentioned, one defector says Trump was right to be tough on North Korea.
"If you're soft towards North Korea, that's lowering yourself from the beginning, so they'll ask for more. It's good that the president was strong against them but we would have to see how much the U.S. could put on the table and restrain the dictator," said Grace Jo, 26, another North Korean defector and college student studying biology.
"The Panmunjom summit was a beautiful and touching moment and it warmed our hearts," said Jo about South Korean President Moon Jae-in and Kim's meeting on April 27. "But we couldn't just look at it nicely because Kim's image may have changed 100% after that and people don't know the hidden side of Kim. So it's very sad. North Korea hasn't changed for the past 70-80 years. That one moment does not change what it is."
Jo escaped with her mother and her sister in 1998 when she was six years old. Her family lived in China for 10 years, hiding from government eyes, but she was sent back to North Korea twice. She later got United Nations protection and entered the U.S. in 2008 with a refugee status.
"As a North Korea defector, [I know that] whatever decision they make, the benefits go to the leaders and the Pyongyang's upper one percent elites. No support goes to the people. I do have hope but I know that it's not going to be met, so it's sad and upsetting."