Reborn in 2010, the AI ​​in my mind is too powerful.

Full-scale chip testing is far more complex than the previous, simpler testing procedures. Previous tests merely verified that the architecture was successful. This time, it requires testing all sorts of complete characterization data.

On Christmas Eve, several top executives of Aurora Group, including Chen Yiran, Tim Cook, Ellison of Oracle, Connolly of Bain Capital, and representatives from Chase Bank, listened to Gao Dewei's formal report at FlowEDA and saw two complete Mocha and Latte chips for the first time.

Unlike the previous small-cell test chips, the computing arrays on these two full-size versions are now complete.

The test chip only has one minimum verifiable unit; the mocha chip is a 16x16 pulsating array slice, and the latte chip is an 8x8 slice.

These two Mocha wafers are nearly three times larger than the test wafers. The 320 square millimeter silicon wafers are densely covered with a complete pulsating array and a ring of SRAM cache blocks around it. The PCIe PHY, four PLLs, and DMA engine are all present.

The latte uses Wanguoyuan's 40-nanometer process, and the transistor itself is larger than that of 28-nanometer, with the full size reaching 480 square millimeters. The gold wire density in the packaged test socket is more than twice that of the test chip.

Gao Dewei placed a mocha sample on the table and began his formal presentation.

"We conducted two weeks of testing, with 30 samples of mocha and 32 samples of latte. Both passed all the tests, and there was no need to modify the metal layer."

The first step is powering on. We added the seven independent power domains of the Mocha chip one by one in sequence: first the core logic voltage, then the interface voltage, then the PLL analog voltage, then the independent power supply for the on-chip SRAM, then the PCIe transceiver, and finally the spare I/O.

When adding each power rail, we simultaneously use a digital multimeter to sample the impedance to ground. If this value drops to a few tenths of an ohm, then we don't need to continue testing, as it indicates a direct internal short circuit.

After the first chip in the Mocha power supply was added, the impedance to ground stabilized within 3% of the value stated in the datasheet. The Latte power supply had five power domains, two fewer than the Mocha, so its deviation was slightly larger, reaching 5.7%, but all were still within the safe range.

Latte's PDK accuracy has always been slightly lower than Samsung's, but this gap has been there since the last round of testing and hasn't widened further, so it's not a big problem.

Next is the clock. Mocha has four PLLs, each operating independently: one supplies the clock to the computing core, one to the storage controller, one to the PCIe, and one to the memory controller.

Latte is not that complicated; it only has two sets because its control logic is much simpler and does not require as many independent clock domains.

Phase-locked loop (PLL) essentially involves feeding a low-frequency reference clock into the PLL and gradually increasing the frequency step by step. After each step, a 0.5-millisecond delay is waited to check if the PLL output has caught up.

Mocha started pulling from 1.2GHz all the way up to 1.6GHz. The entire phase-locked loop curve is very nice, basically linear, with each of the four steps having a phase-locked loop time of less than 0.3 milliseconds.

During the process of upgrading the latte from 500MHz to 800MHz, there was a slight lag around 760MHz, which was 0.15 milliseconds slower than the normal phase-locked loop speed.

This isn't surprising, as a similar issue occurred with the PLL in the previous round of latte test chips, where the locking range was significantly narrowed. While the locking range has been restored after the PDK update, the 0.15ms delay hasn't been completely eliminated.

After some analysis, we concluded that it was likely a voltage-controlled oscillator based on a 40nm process, where the inductor Q value was incorrectly modeled at the low-frequency range.

This can't be changed by modifying the design; we can only wait for WanGuoYuan to upgrade the manufacturing process themselves. Fortunately, this doesn't affect functionality and won't cause it to lose its lock.

The scan chain is relatively simple; it involves JTAG feeding a sequence of 0s and 1s into the registers, letting it pass through all the registers, and then reading it back.

All thirty of the Mocha chips passed the test. Two of the thirty-two Latte chips had slightly higher readback latency (about three clock cycles) near the I/O ring, but no data was lost. This isn't critical; we can monitor it during mass production.

The SRAM part was the most troublesome for us in the last round.

The Mocha test chip had two consecutive weak bits at the low voltage angle, but this time the full-size version passed all three voltage angles. Moreover, the design included a spare line for replacement, so it can be repaired if a problem occurs.

The previous problem with the latte was a bit more complicated, involving SRAM bit flipping at high frequencies. It was later found that the parasitic capacitance parameters of the metal vias in the PDK were off by several femtoseconds. After updating the PDK model and recalibrating it, this time it passed all three voltage angles in the test.

There's another thing I think is worth mentioning. The Mocha and Latte use the same pulse array architecture. The memory interface was designed as a process-independent abstraction layer from the early stages. Only the physical implementation of the transistors was changed on both sides, without altering any logical structure.

Therefore, the SRAM pass rate was much higher than we initially expected, indicating that the architecture itself is not the problem.

When we were working on the process-independent abstraction layer, several people in the team were actually against it, saying that optimizing the performance on both sides separately would be better.

Looking back now, if we hadn't made it an abstraction layer, Mocha and Latte would have had to maintain their own set of control logic for the storage interface. If a bug occurred on either side, we would have to investigate it separately. It might have turned into Mocha fixing problem A and Latte fixing problem B, and the two sides would never be aligned.

With this result, I feel that the abstraction we did back then wasn't in vain.

Functional testing of the computing unit only begins after the preceding electrical verifications have been completed.

We load the pre-stored matrix multiplication vector into the on-chip SRAM and use the DMA engine to drive the computing array. FP32 runs one round, FP16 runs one round, and INT8 runs another round. In each round, we compare it bit by bit with the reference value on the CPU.

摩卡这边FP32的偏差只有0.15%，INT8的吞吐量到了规格书的103%。拿铁FP32偏差0.24%，INT8吞吐量98%，偏差确实比摩卡大了一点，主要是因为40纳米的乘法器电晶体匹配精度天生就比不过28纳米。

However, the deviations of both chips were within the 2% acceptance threshold, so there were no problems.

Then you run the complete network. When convolutional layers and activation functions are combined, the hardware triggers different control flow paths, such as pooling, normalization, and skip connections across layers. Just because matrix multiplication works doesn't mean that all of these are fine.

We loaded a three-layer convolutional network and a small residual network, which were intentionally designed to handle different extreme cases.

The three-layer data structure is relatively dense and continuous, and the data in the on-chip SRAM doesn't need to be swapped in and out very frequently. The residual structure has cross-layer jumps, mainly to test whether the control logic can correctly manage several data streams entering and leaving the computing array simultaneously.

The mean square error between the output and the reference value for Mocha is 3.2e-6, and for Latte it is 4.1e-6.

The PCIe bandwidth measurement involves the connection between the chip and external components.

The Mocha uses a PCIe 3.0 x16 interface, and the actual bidirectional bandwidth of the DMA engine reaches 14.8 GB/s. The theoretical physical layer ceiling is 15.75, but after deducting the link encoding overhead and transaction layer packet headers, the effective bandwidth limit is approximately 14.9.

Mocha's 14.8 is less than a percentage point away from the ceiling, so it's basically not a bottleneck. Latte, on the other hand, uses PCIe 2.0 x8 and reached 3.6GB/s, with a theoretical value of 4.0, which is within the expected range.

For power consumption characterization, we pull the load from 0% to 100% at different ratios, record the current every 10%, and use a thermal imager to scan the surface of the bare die.

The Mocha chip consumes 27.3 watts under full load. In the previous test, the voltage was increased to 0.95V to stabilize the frequency, which increased the power consumption by nearly 10%.

Now that this version has returned to the nominal 0.9V, the power consumption is completely within the 30W budget, and the figure of 27.3 is even slightly lower than the typical value in the datasheet, indicating that the leakage current control of the 28nm process is better than we expected.

The latte consumes 18.7 watts at full load. The devices it's used in don't have very strict power consumption requirements, so achieving this figure using a 40-nanometer process is acceptable.

In the thermal imaging image, the hot spots of both chips are concentrated in the pulsating array area. The latte has a small local hot spot on the side closer to the IO ring. This is the inherent heat generation of the high-current IO driver itself and does not affect the function.

Finally, there's the long-term stress test.

We run the chip at full speed matrix multiplication at an ambient temperature of 85 degrees Celsius, and sample the accuracy and power consumption every fifteen minutes.

Mocha didn't have a single faulty bit for a full four hours.

The latte chip experienced a bit flip around the three-and-a-half-hour mark, but it recovered automatically in the next sampling cycle and did not reproduce upon retesting. The team determined it was a single-event event, where a cosmic ray happened to hit an SRAM memory cell, and it was unrelated to the chip's design.

That's the basic situation. After running nine tests, both the mocha and latte passed, with all deviations within acceptable limits, and no revisions needed. Frankly, this result is better than I expected.

One is Samsung's 28nm, and the other is WanGuoYuan's 40nm; the transistor channel doping, number of metal layers, and photolithography precision are all different. Given so many variables, the fact that the test data for the same pulsating array architecture is so consistent on both sides suggests that the architecture itself has already proven to be correct.

Samsung's packaging line can produce the first batch of mass-produced test wafers in about four weeks, while Wanguoyuan's 40-nanometer line has only been in operation for a short time, so it will take about five weeks.

The design team can freeze the process directly; I've already had them prepare the encryption package.

Chen Yiran waved for Gao Dewei to leave after his report, then looked at the others at the table and asked, "Looking at the current situation, we should be able to officially begin mass production and deployment in the first quarter of next year. Do you have any further questions?"

Connelly wasn't a professional and, after listening to the report, wasn't entirely clear on the chip's performance. Looking at the documents in his hand, he asked, "Mr. Chen, so how far ahead is this chip? Can it completely break away from Nvidia?"

"The closest product on the market is the Tesla K20X based on NVIDIA's Kepler architecture, but it is, after all, a traditional GPU. Our tensor processor, however, is a dedicated tensor processor."

The key difference isn't performance, but rather that our chip is more powerful at the same power consumption. Of course, the current design can only perform inference, not training. However, I already have a complete solution: connect 256 chips into a training cluster, and simultaneously perform gradient aggregation for both forward computation and backpropagation.

Most importantly, there's the synergy between our chip and DeepFlow's native hardware and software capabilities.

Cook, looking at Ellison beside him, asked, "Once chip mass production begins, Apple will need a large number of data centers. Hopefully, Oracle can release its production capacity ahead of schedule. All our data centers are built; we're just waiting for the chips and the data centers."

Ellison nodded: "Don't worry, I'll prioritize supplying Apple. I have another question. Does this mean we have to directly break ties with Nvidia? They've been poaching a lot of people lately. Will we sell our chips externally or use them entirely ourselves? If we sell them externally, we have to consider the possibility that Nvidia will completely side with Google."

Chen Yiran gave a direct conclusion: "Selling is definitely necessary, otherwise our current supply chain is too complete and it would be suspected of being a monopoly. However, we will not sell the training technology, so ideally, Nvidia and AMD chips on the market will be used for training, and our chips will be used for inference."

Of course, if Nvidia really goes to extremes and completely aligns with Google, then we will increase our technological investment in Wanda Fruit to prepare for full supply. There's another issue here: Wanda Fruit's production capacity and technology alone will find it difficult to break through the 28-nanometer barrier in the short term.

So we need to address Samsung's issues and ensure they are completely aligned with our goals.

Besides, we're watching his stock price soar every day thanks to our company, while we don't benefit from it. That won't do!

Connery laughed and said, "Leave it to me. We'll find a way to get them to give up some shares to Aurora Group."

Chen Yiran thought for a moment and said, "Aurora Group is a large company. I left some shares for strategic investors. If Samsung is willing, it can take over some shares. We can have cross-shareholding."

Moreover, they need to come up with cash. Aurora Group will be established for almost a year; they can't have a valuation of 100 billion last year and still be 100 billion this year!