newslounge.co · May 2, 2024

Good Morning. 

Welcome to another edition of newslounge. If you're new here, every week I share what I find interesting in the world of VFX and AI. A little deep dive into what I'm working on, along with the new technologies and workflows. Lets get to it.

On Today’s menu:

• Sam Altman at ETL

• The Future of Filmmaking

• LLama3 and LLava Models in ComfyUI

• The Art of Short-Form Content

• MotionDirector vs CameraCtrl

-Ardy

was this email forwarded to you? you can sign up here.

🚨Just a quick note: You can share this newsletter with your friends using the unique referral link at the bottom of this email. By doing so, you'll earn points that can be exchanged for some cool prizes. Let's spread the word! 😎

HEADLINE

I DON'T CARE IF WE BURN $50 BILLION A YEAR

During his speech at the Entrepreneurial Thought Leader Seminar (ETL) at Stanford University, Sam Altman discussed the significant financial investment required to develop artificial general intelligence (AGI).
He mentioned that the cost for training GPT-3 was around $100 million, and for GPT-4, the cost increased to $400 million due to a tenfold increase in parameters. Despite these high costs, Altman emphasized the importance of continuing investment in AGI development, as he believes the potential societal benefits far outweigh the financial inputs.
He conveyed a commitment to ensuring that OpenAI remains on a trajectory that will eventually generate significant societal value, regardless of the escalating expenses associated with advancing AGI technologies.

The Big Picture:
Altman reiterated OpenAI’s commitment to building general-purpose AI that could benefit all humanity, underscoring the company's foundational goals and vision for the future.

-NL

TRENDING

THE FUTURE OF FILMMAKING

Paul Trillo, a filmmaker, has been exploring the integration of AI tools in the filmmaking process. Here’s a concise breakdown of how he used these tools in his film “Noted To My Future Self,”.

Pre-Production to Post-Production Workflow:

• Background Generation: For each actor's line, Trillo combined the dialogue with keywords to generate thematic backgrounds using the prompt in Automatic1111.

Lighting and Compositing:

• Reference Images: The stable diffusion images created served as lighting references on set.

• Editing Backgrounds (BG): He utilized Photoshop's Generative Fill and Expand to refine these backgrounds.

• Upscaling: Background images were enhanced to 8K using Magnific, KreaAI, and Topaz Labs.

Dynamic Elements and Effects:

• Animation: Using Runway’s Gen-2, Trillo animated elements like bushes and fog.

• Luma Mattes: Animated luma mattes were also created with Gen-2 to simulate dynamic lighting reacting to the environment.

This streamlined approach facilitated a rapid production timeline, allowing shooting, editing, and compositing within ten days. The use of AI effectively bridged the gap from pre-production to post-production, making the integration of visual effects more cohesive and efficient.

-NL

CASE STUDY

BETTER VISUAL DESCRIPTIONS WITH LLAMA3 AND LLAVA IMAGE MODEL

Meta Llama3 was released earlier this month and it is blowing my mind.

✨LLAMA3 is a text-based language model. To fully leverage its capabilities, it should be paired with an image language model such as LLava or Claude3.

✨LLAVA is a multi-purpose visual and language model, great at interpreting images.

How to use it?

• Install Ollama and download the fine-tuned Llama3 model—Trained by the developer of ImpactFrames AI nodes for ComfyUI-

• Install "IF_AI" tools for ComfyUI

• Using the "IF_image_to_prompt" node, you can interpret your input image and extract key descriptions using the LLava image model.

• Using the "IF_prompt_to_prompt" node, you can refine your initial text with LLama3 and add nuanced details.

👽LLama3 final response based on initial image interpretation by LLava model:
Response: Award winning, masterpiece, High detail, Here's a visual prompt for the image:

**Title:** "Epic Nike Billboard"

**Description:** Soar to new heights with this stunning image! Capture the essence of Nike's iconic logo on the rooftop of a majestic skyscraper in Manhattan. With its sleek design and bold typography, this massive billboard stands out against the vibrant urban landscape, radiating energy. **Nike Cover Art Epic **8k Best quality, professional, Portrait, photo shoot,

🙋‍♂️you can download my workflow for incorporating these models.

THE ART OF SHORT-FORM CONTENT ON A SMALL BUDGET

In today’s digital landscape, the ability to create impactful and engaging content quickly and cost-effectively is crucial for creators looking to stand out.
By building a workflow that leverages new technological aids, you can minimize costs and maximize your creative potential, ensuring that you make the most of both your time and resources. That is the way I look at these things!!

🤿Here’s a brief insight into the early stages of a new concept I'm developing. It starts with initial scene setups and effects in Houdini, followed by building characters and design concepts in ComfyUI and Photoshop. I then blend images in ComfyUI to create various compositions.

I trained a camera motion LoRA using ADMotionDirector based on my 3D render and used that to drive my AnimateDiff video.

Here is the GitHub page if you'd like to try it.

Another option would be to use CameraCtrl with AnimateDiff, which allows you to export a camera pose from Blender using Python code as text. You can then use this or the available default camera motions, but be aware that the resolution is somewhat limited.

🚨Here is the workflow by @DREX - [download]

I’ll make a breakdown video once this project is finished!!

-AA

💥One more thing: Please use the poll section below to share your feedback on this case study.

We are seeking:
full-stack software engineer
copywriter
web developer

[email protected]

WANT TO BUILD A NEW AD CAMPAIGN OR LEARN ABOUT AI INTEGRATION?

REPLY TO THIS EMAIL TO SCHEDULE A CALL!!

Earn free gifts 🎁

5 referrals - “Water Bottle Stickers” 🔺15 referrals - “Mystery Box”
25 referrals - “Water Bottle” 🔺 40 referrals - “Nuphy Desk Mat”
60 referrals - “Logitech Mouse”