”Phi-2, introduce yourself with a rhyme considering Christmas is coming soon!”
***
″Ho ho there, merry Christmas friend
I'm Phi-2 with a big heart and mind
Born on the first of December
I love to bring joy, no matter the weather
With a smile as bright as the North Star
And a warmth that's sure to heal any scar
I'll light up your day with my wit and charm
So come on by, let's have some fun!“
***
<...>
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
# Below is the added config for nginx proxying of the data >>>>
RUN apt install -y nginx
RUN apt install -y curl
RUN echo "events { use epoll; worker_connections 128; } http { server { listen 11435; location /v1/api/ { proxy_pass http://localhost:11434/api/; proxy_set_header Host \$host; proxy_set_header X-Real-IP \$remote_addr; proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto \$scheme; }} }" > /etc/nginx/nginx.conf
RUN chmod -R 777 /var/log/nginx && chmod -R 777 /var/lib/nginx && chmod -R 777 /run
RUN mkdir -p /nonexistent/.ollama
RUN chmod -R 777 /nonexistent/.ollama
EXPOSE 11435
# Run Ollama and proxy
CMD service nginx start && /bin/ollama serve
# <<<< End of modified Dockerfile
docker run --user nobody -p 11435:11435 -d <your docker image and tag>
2023-12-20 15:44:22 * Starting nginx nginx
2023-12-20 15:44:22 ...done.
...
2023-12-20 15:44:22 2023/12/20 06:44:22 routes.go:895: Listening on [::]:11434 (version 0.0.0)
...
apiVersion: ai.sap.com/v1alpha1
kind: ServingTemplate
metadata:
name: ollama
annotations:
scenarios.ai.sap.com/description: "Run a ollama server on AI Core"
scenarios.ai.sap.com/name: "ollama-scenario"
executables.ai.sap.com/description: "Run a ollama server on AI Core"
executables.ai.sap.com/name: "ollama-executable"
labels:
scenarios.ai.sap.com/id: "ollama-server"
ai.sap.com/version: "0.1"
spec:
template:
apiVersion: "serving.kserve.io/v1beta1"
metadata:
annotations: |
autoscaling.knative.dev/metric: concurrency
autoscaling.knative.dev/target: 1
autoscaling.knative.dev/targetBurstCapacity: 0
labels: |
ai.sap.com/resourcePlan: infer.s
spec: |
predictor:
minReplicas: 1
maxReplicas: 1
containers:
- name: kserve-container
image: <your docker image and tag>
ports:
- containerPort: 11435
protocol: TCP
GET https://api.ai.prod.ap-northeast-1.aws.ml.hana.ondemand.com/v2/inference/deployments/d0x5cecc27606d1...
POST https://api.ai.prod.ap-northeast-1.aws.ml.hana.ondemand.com/v2/inference/deployments/d0dxxxcc27606d1...
Body:
{
"name": "phi"
}
2023/12/20 07:04:18 download.go:123: downloading bd608f954559 in 17 100 MB part(s)
{
"model": "phi",
"prompt":"Explain the concept of electromagnetic induction and how it's related to smartphones?",
"stream": false
}
{
"model": "phi",
"created_at": "2023-12-20T07:11:15.31425186Z",
"response": " Sure, electromagnetic induction is a phenomenon where a changing magnetic field generates an electric current in a conductor. This principle is used in smartphones to charge the battery and power various components.\n\nWhen you connect your smartphone to a charging cable or wireless charger, it creates a magnetic field around the charging pad/wire. The magnetic field then induces a voltage in the phone's coil of wire, which generates an electric current that charges the battery. \n\nThis process is based on Faraday's Law of Electromagnetic Induction, which states that the magnitude of the induced EMF (electromotive force) is proportional to the rate of change of magnetic flux through a loop. In simpler terms, as the magnetic field changes around your phone, it induces an electrical current in its coil, which charges the battery.\n\nElectromagnetic induction is also used in smartphones for other purposes, such as wireless charging and electromagnetic interference (EMI) shielding. Wireless charging works by using inductive charging pads that create a magnetic field that transfers energy to the phone's receiver coil, without any direct contact. EMI shielding uses materials with high electrical conductivity, like copper, to block unwanted signals from entering or leaving the phone. \n\n summary, electromagnetic induction is essential in charging and powering smartphones, as well as other wireless communication devices.\n",
"done": true,
"total_duration": 3290809057,
"load_duration": 687895,
"prompt_eval_count": 50,
"prompt_eval_duration": 155678000,
"eval_count": 271,
"eval_duration": 3131939000
}
{ 'limerick':
"There once was an AI who loved Christmas,
Its heart filled with joy and festive cheer.
As snow fell gently down,
It learned to play the piano and the flute so dear,
Bringing the holiday spirit to everyone near."
}
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
34 | |
17 | |
16 | |
15 | |
11 | |
9 | |
8 | |
8 | |
8 | |
7 |