Reborn in 2010, the AI ​​in my mind is too powerful.

In the following days, Chen Yiran continued to stay in his apartment writing code.

I'll be Jasmine's human keyboard, pushing the buttons for an hour or two every day. The framework for DeepFlow's automatic differentiation engine is already built, but the backpropagation part still hasn't gotten going. Jasmine modified the activation function implementation yesterday, and she has to keep tweaking it today.

[On line 12, replace sigmoid with tanh, and change the text in parentheses to hidden_output.]

Chen Yiran typed it out, and TextMate's code completion did pop up, but it only showed irrelevant things, so he had to type it out letter by letter.

Then add a line below, `np.dot`, whose parameters are the transpose of `error` and `hidden_output`.

"Wait—how do you write transpose again?"

【T is added after the variable.】

He finished typing it, ran it, and an error message popped up in the Terminal: Dimension mismatch.

The shape of `hidden_output` should be 64 by 1; you need to add a `reshape` keyword before it.

Which line?

[Line 47.]

He scrolled up to line 47 and, sure enough, it was missing a `reshape` statement. He added it, ran it again, but it still gave an error. This time, the problem was with `learning_rate`; it was too large, causing gradient explosion.

Chen Yiran leaned back in his chair and sighed. This kind of parameter tuning was the most annoying thing. Either the logic was wrong or the values ​​were incorrect. Changing one thing would cause a whole series of problems.

Jasmine understands all the theories, but he still has to run it again and again, check the error messages, fix it, and run it again.

After more than an hour of struggling, it finally got running.

But the loss is dropping too slowly, almost horizontally, and it only decreased by a few tenths of a percent after two hundred epochs. At this rate, it will take until next year.

[This might be a problem with weight initialization. Currently, all-zero initialization is being used, which causes all hidden layer neurons to output the same value, resulting in identical gradients during backpropagation.]

Speak like a human.

Change `np.zeros` to `np.random.randn` in lines 23 and 25, and divide by the square root of `hidden_size`.

"Should I just type sqrt like that?"

[np.sqrt, with the hidden_size specified in parentheses.]

After making changes to both places, I tried running it. This time, the loss curve was significantly steeper, but on the fiftieth epoch, a NaN suddenly appeared, ruining the entire training.

"It exploded again."

The learning rate is still too high. 0.1 is too high for random initialization; let's try lowering it to 0.01.

"Didn't you just say 0.1 was too big? How come it's too big now?"

[Previously, with zero initialization and a small gradient, it barely ran with a gradient of 0.1. Now, with random initialization, the initial gradient is larger, and it passes with a gradient of 0.1.]

"Can't you just do it all at once? Okay, that's enough for today, my brain's about to explode. Seriously, can't you think it through before you let me play?"

[You mean I can access your brain more deeply, without having to keep it in a low-level state? If so, I can fully simulate most of the operational scenarios. Please prepare supplies and ice in advance.]

"No! I was just saying, it's fine this way."

Ignoring Jasmine, who was annoyingly honest every day, Chen Yiran called and ordered a pizza.

Just as I was about to hang up the phone, Xu Li sent me a text message: "The bill of lading has arrived! The freight forwarder just sent it to my email! I also sent you a copy; you should show this to people in China."

Chen Yiran opened her Gmail on her computer and, sure enough, found a forwarded email from Global Auto Shipping, titled "Bill of Lading - 2x Cayenne - Tianjin." She glanced at the attached bill of lading—the vessel name was COSCO Development, voyage number 0042W, and the VINs of both Cayennes matched. The ship departed from Newark on February 17th, which was Wednesday, and the ETA arrival time at Tianjin Port was approximately four weeks later.

Chen Yiran quickly forwarded the message to Manager Wang, and then told Xu Li.

Xu Li replied with a hand-rubbing emoji.

He put down his phone and casually scrolled through Renren.com.

The pizza arrived quickly, and he continued surfing Renren at a high intensity while eating it.

He's recently become an ELI legend, scoring a perfect TOEFL score just three weeks into his freshman year. He doesn't have to attend classes, spends his days sleeping, scrolling through Renren (a Chinese social networking site), and replying to messages everywhere – he's incredibly bored.

Many friends who were very close in a past life, though not as close in this life, have reconnected online.

After finishing a pizza, Chen Yiran was still a little hungry, so she drove to ACME to do some shopping.

Jasmine's presence allowed Chen Yiran to indulge in binge eating without worry, since at worst she could just rest her head on ice and let Jasmine run wild for a while.

Chocolate, chocolate ice cream, chocolate milk. A variety of high-calorie foods, and countless fruits, filled my shopping cart.

When Chen Yiran got home, she suddenly felt that she had become a bit lazy. For the past two days, apart from working hard for one or two hours each day under Jasmine's supervision, she had spent the rest of her time browsing the web, watching movies, or eating.

No, I need to get down to business.

I might as well open Google and check out the application requirements for Stanford University. The little script I posted on GitHub should be enough to avoid the hassle of the SAT. Geniuses always have some privileges. But I still need to check the specific requirements. At least, I should look at the application deadlines and such.

I looked on the official website, clicked on Computer Science, and then Admission.

Fall Quarter 2011: Application deadline December 1, 2010.

Spring Quarter 2011: No admission offered.

Chen Yiran's hand stopped on the touchpad.

WTF, WTH, HS!

Chen Yiran immediately dropped the American-style three-part punchline, and quickly searched the official website carefully, only finding one line of text in the Q&A section.

Are first-time, first-year students accepted for terms other than the fall? No

They only harvest in the fall, how disgusting!

I never played high-level games in my past life, so I had no idea that Stanford didn't have spring semester admissions.

Even his ELI professors didn't tell him that, and they even gave him advice on how to maintain his F1 status.

This shows that even the professor hasn't played high-level games. Is the University of Delaware really not good enough?

After calming down for a while, Chen Yiran opened Google.

Search for "top computer science programs spring admission 2011" and press Enter.

The first ranking from US News, okay, let's take a look at this one first.

A search on the official websites of the top-ranked universities reveals that Carnegie Mellon University only offers fall semester admission, as do MIT and Caltech.

Berkeley and Santa Clara are both only in autumn.

After searching for a long time, the only unexpected surprise was the University of Champaign. The University of Champaign's fall recruitment is not even over yet, with the deadline being March 1st.

His CS major was also ranked very high, fifth in the United States.

However, the nearest major city in central Illinois is Chicago, a two-hour drive away. As for Silicon Valley? It's a four- or five-hour flight, with at least one transfer in between.

Chen Yiran hesitated a bit; he wasn't actually that concerned about university courses. With Jasmine around, his learning ability was phenomenal.

Going to school is mainly about resources—the resources around you, the resources from your professors. Although he hadn't started a business in his previous life, he had read a lot of wish-fulfillment novels.

Based on his extensive experience in creating content for the novel industry, he believes that Silicon Valley's location advantage is the most important factor.

So his initial target was Stanford, or Berkeley, Santa Clara, or any university around Silicon Valley.

The University of Champaign's appeal to him might just be its catchy name.

Located next to Silicon Valley, you can bump into startups just by stepping outside, and hear people discussing tech trends as soon as you enter a coffee shop. He has no idea what's around the University of Champaign.

A quick search reveals a staggering population of 7, almost twice the population of Newark, where the University of Delaware is located.

This is just a slightly larger village; you can't go any further.

He closed all the tabs, leaving only the half-open TextMate project window on the screen.

Chen Yiran pushed her chair back and stared at the ceiling for two minutes.

In the end, I couldn't think of any good solutions.

It's simply a matter of enrolling in Delaware in September, then getting a good GPA and transferring later.

Or you could spend a little more money and just stay at ELI until next year.

ELI costs over three thousand per period, and about eighteen thousand for six periods a year. The parallel car business is still easy to cover.

Or you could simply go back to China for a year, but that's not as good as staying in the US, where you could at least do some parallel import business or something.

"Jasmine, that means we might have more time now, so let's do some more impressive things."

【it is good. 】